2024-09-19T09:12:53.405 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T09:12:53.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:12:53.525 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911911' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHctCwfsHhvEqlXX4NDTOWNr2x0qZNUGuxC27De9jv+JRqi87oem6zpcq2dsoYDsBkDk8TLbVGPqppSehGsGiM= smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWYPKDLcAkypU0di/SKbHbMS9i/6YZlnyyIvpIL64EE3fd7KvkX9KqocFtvBinT2G4Y6fm0slWvzHW8jW/xELU= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA2116LB15BCjZIJIsoubvRq/nz62KkhhcNh0b13PdHugDpHcc6Gvb2P1JfPpIQYQES1nTL8RzKonE1IY6Y8I6w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T09:12:53.526 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T09:12:53.526 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T09:12:53.526 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T09:12:53.528 INFO:teuthology.task.internal:Checking packages... 2024-09-19T09:12:53.549 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T09:12:53.549 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T09:12:53.549 INFO:teuthology.packaging:ref: None 2024-09-19T09:12:53.549 INFO:teuthology.packaging:tag: None 2024-09-19T09:12:53.549 INFO:teuthology.packaging:branch: squid 2024-09-19T09:12:53.549 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:12:53.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T09:12:53.754 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T09:12:53.756 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T09:12:53.763 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T09:12:53.763 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T09:12:53.801 INFO:teuthology.task.internal:Saving configuration 2024-09-19T09:12:53.821 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T09:12:53.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T09:12:53.882 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:08:04.371844', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAHctCwfsHhvEqlXX4NDTOWNr2x0qZNUGuxC27De9jv+JRqi87oem6zpcq2dsoYDsBkDk8TLbVGPqppSehGsGiM='} 2024-09-19T09:12:53.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:08:04.373366', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWYPKDLcAkypU0di/SKbHbMS9i/6YZlnyyIvpIL64EE3fd7KvkX9KqocFtvBinT2G4Y6fm0slWvzHW8jW/xELU='} 2024-09-19T09:12:53.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:08:04.374540', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA2116LB15BCjZIJIsoubvRq/nz62KkhhcNh0b13PdHugDpHcc6Gvb2P1JfPpIQYQES1nTL8RzKonE1IY6Y8I6w='} 2024-09-19T09:12:53.923 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T09:12:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T09:12:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T09:12:53.930 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T09:12:53.930 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T09:12:54.041 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5aedb6fbe0>, signals=[15]) 2024-09-19T09:12:54.041 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T09:12:54.047 INFO:teuthology.task.internal:Opening connections... 2024-09-19T09:12:54.047 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:12:54.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:12:54.125 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T09:12:54.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:12:54.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-09-19T09:12:54.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:12:54.276 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T09:12:54.282 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-09-19T09:12:54.300 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-09-19T09:12:54.301 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-09-19T09:12:54.359 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-09-19T09:12:54.359 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-09-19T09:12:54.359 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:12:54.360 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:12:54.361 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-09-19T09:12:54.386 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-09-19T09:12:54.404 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-09-19T09:12:54.405 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-09-19T09:12:54.463 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-09-19T09:12:54.463 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-09-19T09:12:54.463 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-09-19T09:12:54.463 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:12:54.463 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:12:54.464 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:12:54.464 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-09-19T09:12:54.486 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-09-19T09:12:54.504 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-09-19T09:12:54.505 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:12:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:12:54.564 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:12:54.564 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:12:54.564 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:12:54.564 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:12:54.564 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-09-19T09:12:54.586 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T09:12:54.596 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T09:12:54.605 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T09:12:54.605 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:12:54.608 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:12:54.610 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:12:54.627 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T09:12:54.633 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T09:12:54.633 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:12:54.666 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:12:54.668 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:12:54.695 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T09:12:54.769 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T09:12:54.783 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T09:12:54.783 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T09:12:54.783 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T09:12:54.784 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:12:54.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:12:54.801 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-19T09:12:54.859 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-503.el9.x86_64 2024-09-19T09:12:54.860 INFO:teuthology.task.kernel:Running kernel on smithi028: 5.14.0-503.el9.x86_64 2024-09-19T09:12:54.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum install -y kernel 2024-09-19T09:13:00.002 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.3 MB 00:03 2024-09-19T09:13:01.802 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-19T09:13:07.254 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-19T09:13:08.821 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-19T09:13:09.975 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T09:13:15.505 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T09:13:17.512 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T09:13:17.512 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T09:13:17.608 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T09:13:17.613 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-09-19T09:13:17.614 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:13:17.615 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 79 M 2024-09-19T09:13:17.615 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 126 M 2024-09-19T09:13:17.615 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T09:13:18.232 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 63 kB/s | 36 kB 00:00 2024-09-19T09:13:19.557 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 8.3 MB/s | 16 MB 00:01 2024-09-19T09:13:20.232 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-09-19T09:13:20.657 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-19T09:13:20.658 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:13:20.658 INFO:teuthology.orchestra.run.smithi028.stdout:Total 26 MB/s | 79 MB 00:03 2024-09-19T09:13:21.368 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T09:13:21.735 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T09:13:21.735 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T09:13:22.716 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T09:13:22.717 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T09:13:25.263 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T09:13:25.919 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T09:13:26.000 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:13:28.742 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:13:28.869 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:13:33.999 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:13:34.044 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:13:39.188 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:14:17.983 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:14:18.024 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:14:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:14:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T09:14:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:14:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:14:19.201 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:14:19.201 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:14:19.201 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T09:14:19.201 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.202 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.202 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.202 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.202 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:14:19.202 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T09:14:19.526 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T09:14:19.654 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.654 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T09:14:19.654 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T09:14:19.655 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T09:14:19.655 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T09:14:19.655 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T09:14:19.655 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi028.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T09:14:19.655 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T09:14:19.713 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T09:14:19.713 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T09:14:19.713 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -qi grub2-tools 2024-09-19T09:14:19.754 INFO:teuthology.orchestra.run.smithi028.stdout:Name : grub2-tools 2024-09-19T09:14:19.754 INFO:teuthology.orchestra.run.smithi028.stdout:Epoch : 1 2024-09-19T09:14:19.754 INFO:teuthology.orchestra.run.smithi028.stdout:Version : 2.06 2024-09-19T09:14:19.754 INFO:teuthology.orchestra.run.smithi028.stdout:Release : 82.el9 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Architecture: x86_64 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Group : Unspecified 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Size : 8274670 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:License : GPLv3+ 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Packager : builder@centos.org 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Vendor : CentOS 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T09:14:19.755 INFO:teuthology.orchestra.run.smithi028.stdout:Summary : Support tools for GRUB. 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:Description : 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:hardware devices. 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:14:19.756 INFO:teuthology.orchestra.run.smithi028.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T09:14:19.757 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T09:14:19.757 INFO:teuthology.task.kernel:Updating grub on smithi028 to boot 5.14.0-508.el9.x86_64 2024-09-19T09:14:19.757 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T09:14:20.408 INFO:teuthology.orchestra.run.smithi028.stderr:Generating grub configuration file ... 2024-09-19T09:14:21.764 INFO:teuthology.orchestra.run.smithi028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T09:14:21.797 INFO:teuthology.orchestra.run.smithi028.stderr:done 2024-09-19T09:14:21.799 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-09-19T09:14:21.815 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.aUNYBoNLyn 2024-09-19T09:14:21.815 DEBUG:teuthology.orchestra.run.smithi028:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aUNYBoNLyn 2024-09-19T09:14:21.887 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.aUNYBoNLyn 2024-09-19T09:14:22.027 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.aUNYBoNLyn is 6KB 2024-09-19T09:14:22.076 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.aUNYBoNLyn 2024-09-19T09:14:22.091 DEBUG:teuthology.orchestra.run.smithi028:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T09:14:22.153 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T09:14:22.154 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T09:14:22.154 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T09:14:22.154 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T09:14:22.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T09:14:22.304 DEBUG:teuthology.orchestra.run.smithi028:> sudo shutdown -r now 2024-09-19T09:14:52.308 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T09:14:52.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:14:52.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:10.781 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-09-19T09:15:19.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:15:19.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:38.177 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-09-19T09:15:50.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:15:50.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:52.250 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-19T09:15:52.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:15:52.672 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T09:15:52.672 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-19T09:15:52.687 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-508.el9.x86_64 2024-09-19T09:15:52.687 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T09:15:52.687 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T09:15:52.687 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T09:15:53.689 DEBUG:teuthology.parallel:result is None 2024-09-19T09:15:53.689 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T09:15:53.699 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T09:15:53.700 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:15:53.702 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:15:53.704 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:15:53.723 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T09:15:53.733 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T09:15:53.746 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T09:15:53.746 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:15:53.758 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:15:53.763 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:15:53.884 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T09:15:53.894 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T09:15:53.894 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:15:53.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:15:54.084 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:15:54.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:15:54.101 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:15:54.104 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:15:54.137 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:15:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.191 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.193 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.249 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:15:54.250 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T09:15:54.442 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T09:15:54.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:15:54.445 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:15:54.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:15:54.479 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T09:15:54.489 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T09:15:54.489 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:15:54.513 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:15:54.517 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:15:54.538 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:54.610 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:54.655 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:54.738 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:54.778 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:15:54.778 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:15:54.861 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:54.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:54.960 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:55.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:55.091 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:15:55.091 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:15:55.159 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:55.205 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:15:55.268 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:55.313 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:15:55.358 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:15:55.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:15:55.426 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-09-19T09:15:55.428 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-09-19T09:15:55.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-09-19T09:15:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:15:55.477 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:15:55.506 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:15:55.970 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T09:15:55.979 INFO:teuthology.task.internal:Starting timer... 2024-09-19T09:15:55.980 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T09:15:56.072 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T09:15:56.115 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T09:15:56.116 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-09-19T09:15:56.197 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-09-19T09:15:56.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-09-19T09:15:56.252 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-09-19T09:15:56.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-09-19T09:15:56.309 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-09-19T09:15:56.311 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T09:15:56.311 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-09-19T09:15:56.331 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-09-19T09:15:56.332 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-09-19T09:15:56.360 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-09-19T09:15:56.360 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-09-19T09:15:56.391 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-09-19T09:15:56.391 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive'} 2024-09-19T09:15:56.391 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:15:56.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:15:56.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:15:56.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:15:56.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:15:56.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:15:56.476 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T09:15:56.477 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:15:56.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:15:56.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:15:56.605 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T09:15:56.618 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T09:15:56.629 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T09:15:56.630 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi038.front.sepia.ceph.com,smithi078.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T09:20:01.322 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com')] 2024-09-19T09:20:01.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:20:01.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:20:01.429 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-19T09:20:01.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T09:20:01.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T09:20:01.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:20:01.600 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-19T09:20:01.679 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T09:20:01.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-19T09:20:01.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:20:01.761 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-19T09:20:01.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-19T09:20:01.845 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T09:20:01.855 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T09:20:01.856 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:20:01.856 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:20:01.859 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:20:01.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:20:01.861 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:20:01.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:20:01.891 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:20:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:20:01.907 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:20:01.908 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:20:01.914 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:20:01.929 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:20:01.934 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-09-19T09:20:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-09-19T09:20:01.946 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-09-19T09:20:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-09-19T09:20:01.955 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-09-19T09:20:01.961 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:20:01.962 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:20:01.966 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-09-19T09:20:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:20:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:20:01.982 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:20:01.997 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:20:02.032 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:20:02.034 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.037 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.038 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:20:02.038 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-19T09:20:02.038 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.038 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.039 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.039 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.056 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:20:02.059 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:20:02.059 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-19T09:20:02.059 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.059 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.060 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.060 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:20:02.061 INFO:teuthology.run_tasks:Running task install... 2024-09-19T09:20:02.068 DEBUG:teuthology.task.install:project ceph 2024-09-19T09:20:02.068 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T09:20:02.068 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T09:20:02.068 INFO:teuthology.task.install:Using flavor: default 2024-09-19T09:20:02.074 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T09:20:02.074 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T09:20:02.075 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:20:02.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:20:02.077 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:20:02.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:20:02.079 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:20:02.079 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:20:02.222 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:20:02.223 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:20:02.231 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:20:02.231 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:20:02.237 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:20:02.237 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:20:02.333 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:20:02.333 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:20:02.334 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:20:02.339 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:20:02.340 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:20:02.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:20:02.347 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:20:02.347 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:20:02.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:20:02.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:20:02.360 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:20:02.365 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:20:02.366 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:20:02.374 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:20:02.375 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:20:02.427 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:20:02.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:20:02.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:20:02.505 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:20:02.506 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:20:02.517 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:20:02.570 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-09-19T09:20:02.570 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-09-19T09:20:02.572 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-09-19T09:20:02.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-09-19T09:20:02.581 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-09-19T09:20:02.583 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-09-19T09:20:02.797 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-09-19T09:20:02.813 INFO:teuthology.orchestra.run.smithi038.stdout:45 files removed 2024-09-19T09:20:02.818 INFO:teuthology.orchestra.run.smithi028.stdout:45 files removed 2024-09-19T09:20:02.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:20:02.840 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:20:02.846 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:20:03.359 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-19T09:20:03.409 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 295 kB/s | 84 kB 00:00 2024-09-19T09:20:03.414 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-19T09:20:03.627 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-19T09:20:03.702 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T09:20:03.706 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T09:20:03.910 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T09:20:03.961 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-19T09:20:03.990 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-19T09:20:04.519 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-19T09:20:05.098 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.3 MB 00:01 2024-09-19T09:20:05.728 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-19T09:20:07.116 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-19T09:20:07.682 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-19T09:20:12.179 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-19T09:20:13.275 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-19T09:20:13.813 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-19T09:20:14.851 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-19T09:20:14.943 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T09:20:15.630 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-19T09:20:20.557 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-19T09:20:21.277 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T09:20:22.768 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:22.778 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:22.781 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:20:22.781 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:20:22.782 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:20:22.783 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:20:22.785 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:20:22.787 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:20:22.788 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:20:22.855 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-19T09:20:22.877 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:20:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:20:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:20:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:20:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:20:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:20:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:20:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:20:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:20:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:20:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:20:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:20:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:20:22.890 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-19T09:20:22.891 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T09:20:22.891 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-09-19T09:20:22.891 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 8 Packages 2024-09-19T09:20:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:20:22.891 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 226 M 2024-09-19T09:20:22.892 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-19T09:20:23.546 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:23.547 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:23.559 INFO:teuthology.orchestra.run.smithi028.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:20:23.559 INFO:teuthology.orchestra.run.smithi028.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:20:23.560 INFO:teuthology.orchestra.run.smithi028.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:20:23.561 INFO:teuthology.orchestra.run.smithi028.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:20:23.563 INFO:teuthology.orchestra.run.smithi028.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:20:23.565 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:20:23.566 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:20:23.632 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T09:20:23.653 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:20:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:20:23.655 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:20:23.656 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:20:23.657 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:20:23.658 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:20:23.659 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:20:23.660 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:20:23.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:20:23.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:20:23.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:20:23.664 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:20:23.665 INFO:teuthology.orchestra.run.smithi028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout:Install 148 Packages 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 8 Packages 2024-09-19T09:20:23.666 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:20:23.667 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 226 M 2024-09-19T09:20:23.667 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T09:20:23.768 INFO:teuthology.orchestra.run.smithi038.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T09:20:24.017 INFO:teuthology.orchestra.run.smithi038.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.5 MB/s | 896 kB 00:00 2024-09-19T09:20:24.087 INFO:teuthology.orchestra.run.smithi028.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 34 kB/s | 6.6 kB 00:00 2024-09-19T09:20:24.101 INFO:teuthology.orchestra.run.smithi038.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T09:20:24.126 INFO:teuthology.orchestra.run.smithi038.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.3 MB/s | 145 kB 00:00 2024-09-19T09:20:24.210 INFO:teuthology.orchestra.run.smithi038.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 22 MB/s | 2.4 MB 00:00 2024-09-19T09:20:24.360 INFO:teuthology.orchestra.run.smithi038.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-19T09:20:24.637 INFO:teuthology.orchestra.run.smithi028.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.6 MB/s | 896 kB 00:00 2024-09-19T09:20:24.704 INFO:teuthology.orchestra.run.smithi028.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T09:20:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 9.0 MB/s | 4.7 MB 00:00 2024-09-19T09:20:24.929 INFO:teuthology.orchestra.run.smithi028.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-09-19T09:20:25.147 INFO:teuthology.orchestra.run.smithi028.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.0 MB/s | 1.5 MB 00:00 2024-09-19T09:20:25.392 INFO:teuthology.orchestra.run.smithi038.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 14 MB/s | 21 MB 00:01 2024-09-19T09:20:25.392 INFO:teuthology.orchestra.run.smithi038.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 432 kB/s | 25 kB 00:00 2024-09-19T09:20:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:19 2024-09-19T09:20:25.555 INFO:teuthology.orchestra.run.smithi028.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 2.8 MB/s | 2.4 MB 00:00 2024-09-19T09:20:25.672 INFO:teuthology.orchestra.run.smithi028.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 9.0 MB/s | 4.7 MB 00:00 2024-09-19T09:20:25.761 INFO:teuthology.orchestra.run.smithi038.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 17 MB 00:01 2024-09-19T09:20:25.878 INFO:teuthology.orchestra.run.smithi038.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 9.3 MB/s | 11 MB 00:01 2024-09-19T09:20:25.903 INFO:teuthology.orchestra.run.smithi038.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.6 MB/s | 228 kB 00:00 2024-09-19T09:20:25.959 INFO:teuthology.orchestra.run.smithi038.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 337 kB/s | 27 kB 00:00 2024-09-19T09:20:25.987 INFO:teuthology.orchestra.run.smithi038.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 378 kB/s | 32 kB 00:00 2024-09-19T09:20:26.079 INFO:teuthology.orchestra.run.smithi038.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 6.1 MB/s | 751 kB 00:00 2024-09-19T09:20:26.171 INFO:teuthology.orchestra.run.smithi038.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.4 MB/s | 127 kB 00:00 2024-09-19T09:20:26.238 INFO:teuthology.orchestra.run.smithi038.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T09:20:26.380 INFO:teuthology.orchestra.run.smithi028.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-19T09:20:26.447 INFO:teuthology.orchestra.run.smithi028.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 378 kB/s | 25 kB 00:00 2024-09-19T09:20:26.530 INFO:teuthology.orchestra.run.smithi038.stdout:(18/156): libcephsqlite-19.1.1-361.g2f599a46.el 301 kB/s | 163 kB 00:00 2024-09-19T09:20:26.597 INFO:teuthology.orchestra.run.smithi028.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 11 MB/s | 11 MB 00:00 2024-09-19T09:20:26.614 INFO:teuthology.orchestra.run.smithi038.stdout:(19/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-19T09:20:26.631 INFO:teuthology.orchestra.run.smithi038.stdout:(20/156): python3-ceph-argparse-19.1.1-361.g2f5 449 kB/s | 45 kB 00:00 2024-09-19T09:20:26.656 INFO:teuthology.orchestra.run.smithi028.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.9 MB/s | 228 kB 00:00 2024-09-19T09:20:26.673 INFO:teuthology.orchestra.run.smithi038.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T09:20:26.699 INFO:teuthology.orchestra.run.smithi038.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T09:20:26.714 INFO:teuthology.orchestra.run.smithi028.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 462 kB/s | 27 kB 00:00 2024-09-19T09:20:26.748 INFO:teuthology.orchestra.run.smithi038.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.1 MB/s | 320 kB 00:00 2024-09-19T09:20:26.773 INFO:teuthology.orchestra.run.smithi028.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 544 kB/s | 32 kB 00:00 2024-09-19T09:20:26.774 INFO:teuthology.orchestra.run.smithi038.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T09:20:26.807 INFO:teuthology.orchestra.run.smithi038.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T09:20:26.857 INFO:teuthology.orchestra.run.smithi038.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-19T09:20:26.898 INFO:teuthology.orchestra.run.smithi028.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.9 MB/s | 751 kB 00:00 2024-09-19T09:20:26.941 INFO:teuthology.orchestra.run.smithi038.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T09:20:26.982 INFO:teuthology.orchestra.run.smithi028.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.9 MB/s | 163 kB 00:00 2024-09-19T09:20:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 382 kB/s | 32 kB 00:00 2024-09-19T09:20:27.057 INFO:teuthology.orchestra.run.smithi028.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.6 MB/s | 127 kB 00:00 2024-09-19T09:20:27.108 INFO:teuthology.orchestra.run.smithi038.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-19T09:20:27.124 INFO:teuthology.orchestra.run.smithi028.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T09:20:27.134 INFO:teuthology.orchestra.run.smithi038.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.3 MB/s | 146 kB 00:00 2024-09-19T09:20:27.519 INFO:teuthology.orchestra.run.smithi038.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 8.7 MB/s | 3.6 MB 00:00 2024-09-19T09:20:27.618 INFO:teuthology.orchestra.run.smithi038.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.4 MB/s | 249 kB 00:00 2024-09-19T09:20:27.702 INFO:teuthology.orchestra.run.smithi038.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 592 kB/s | 49 kB 00:00 2024-09-19T09:20:27.775 INFO:teuthology.orchestra.run.smithi028.stdout:(18/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.2 MB/s | 5.3 MB 00:00 2024-09-19T09:20:27.808 INFO:teuthology.orchestra.run.smithi038.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-19T09:20:27.826 INFO:teuthology.orchestra.run.smithi028.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 908 kB/s | 45 kB 00:00 2024-09-19T09:20:27.829 INFO:teuthology.orchestra.run.smithi038.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 131 kB/s | 16 kB 00:00 2024-09-19T09:20:27.869 INFO:teuthology.orchestra.run.smithi038.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T09:20:27.883 INFO:teuthology.orchestra.run.smithi028.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T09:20:27.919 INFO:teuthology.orchestra.run.smithi038.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-19T09:20:27.942 INFO:teuthology.orchestra.run.smithi028.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.7 MB/s | 161 kB 00:00 2024-09-19T09:20:27.944 INFO:teuthology.orchestra.run.smithi038.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 243 kB/s | 18 kB 00:00 2024-09-19T09:20:28.009 INFO:teuthology.orchestra.run.smithi028.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T09:20:28.078 INFO:teuthology.orchestra.run.smithi038.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 502 kB/s | 80 kB 00:00 2024-09-19T09:20:28.162 INFO:teuthology.orchestra.run.smithi038.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T09:20:28.185 INFO:teuthology.orchestra.run.smithi028.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 300 kB 00:00 2024-09-19T09:20:28.234 INFO:teuthology.orchestra.run.smithi028.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T09:20:28.293 INFO:teuthology.orchestra.run.smithi028.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T09:20:28.543 INFO:teuthology.orchestra.run.smithi028.stdout:(26/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-19T09:20:28.638 INFO:teuthology.orchestra.run.smithi038.stdout:(41/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 14 MB/s | 49 MB 00:03 2024-09-19T09:20:28.663 INFO:teuthology.orchestra.run.smithi038.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-19T09:20:28.688 INFO:teuthology.orchestra.run.smithi038.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 482 kB/s | 253 kB 00:00 2024-09-19T09:20:28.719 INFO:teuthology.orchestra.run.smithi028.stdout:(27/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 5.3 MB/s | 17 MB 00:03 2024-09-19T09:20:28.739 INFO:teuthology.orchestra.run.smithi038.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T09:20:28.744 INFO:teuthology.orchestra.run.smithi028.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 853 kB/s | 171 kB 00:00 2024-09-19T09:20:28.763 INFO:teuthology.orchestra.run.smithi038.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-19T09:20:28.769 INFO:teuthology.orchestra.run.smithi028.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 636 kB/s | 32 kB 00:00 2024-09-19T09:20:28.789 INFO:teuthology.orchestra.run.smithi038.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T09:20:28.794 INFO:teuthology.orchestra.run.smithi028.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T09:20:28.839 INFO:teuthology.orchestra.run.smithi038.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-19T09:20:28.931 INFO:teuthology.orchestra.run.smithi038.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-09-19T09:20:28.982 INFO:teuthology.orchestra.run.smithi038.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-19T09:20:29.046 INFO:teuthology.orchestra.run.smithi038.stdout:(50/156): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-09-19T09:20:29.066 INFO:teuthology.orchestra.run.smithi038.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-19T09:20:29.108 INFO:teuthology.orchestra.run.smithi038.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 784 kB/s | 33 kB 00:00 2024-09-19T09:20:29.133 INFO:teuthology.orchestra.run.smithi038.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.5 MB/s | 313 kB 00:00 2024-09-19T09:20:29.183 INFO:teuthology.orchestra.run.smithi038.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-19T09:20:29.217 INFO:teuthology.orchestra.run.smithi038.stdout:(55/156): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-19T09:20:29.242 INFO:teuthology.orchestra.run.smithi038.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-19T09:20:29.276 INFO:teuthology.orchestra.run.smithi038.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T09:20:29.301 INFO:teuthology.orchestra.run.smithi038.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-19T09:20:29.418 INFO:teuthology.orchestra.run.smithi038.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-19T09:20:29.443 INFO:teuthology.orchestra.run.smithi038.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-19T09:20:29.469 INFO:teuthology.orchestra.run.smithi038.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-19T09:20:29.494 INFO:teuthology.orchestra.run.smithi038.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T09:20:29.494 INFO:teuthology.orchestra.run.smithi028.stdout:(31/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:03 2024-09-19T09:20:29.552 INFO:teuthology.orchestra.run.smithi038.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T09:20:29.553 INFO:teuthology.orchestra.run.smithi028.stdout:(32/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 4.5 MB/s | 3.6 MB 00:00 2024-09-19T09:20:29.577 INFO:teuthology.orchestra.run.smithi038.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T09:20:29.578 INFO:teuthology.orchestra.run.smithi028.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T09:20:29.603 INFO:teuthology.orchestra.run.smithi038.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-19T09:20:29.603 INFO:teuthology.orchestra.run.smithi028.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 992 kB/s | 49 kB 00:00 2024-09-19T09:20:29.629 INFO:teuthology.orchestra.run.smithi028.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T09:20:29.662 INFO:teuthology.orchestra.run.smithi028.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T09:20:29.754 INFO:teuthology.orchestra.run.smithi028.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 7.7 MB/s | 7.4 MB 00:00 2024-09-19T09:20:29.770 INFO:teuthology.orchestra.run.smithi038.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-09-19T09:20:29.780 INFO:teuthology.orchestra.run.smithi028.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 156 kB/s | 18 kB 00:00 2024-09-19T09:20:29.813 INFO:teuthology.orchestra.run.smithi028.stdout:(39/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-19T09:20:29.961 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-19T09:20:29.972 INFO:teuthology.orchestra.run.smithi028.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-19T09:20:30.039 INFO:teuthology.orchestra.run.smithi028.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-19T09:20:30.081 INFO:teuthology.orchestra.run.smithi028.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-19T09:20:30.106 INFO:teuthology.orchestra.run.smithi028.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-09-19T09:20:30.112 INFO:teuthology.orchestra.run.smithi038.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-19T09:20:30.156 INFO:teuthology.orchestra.run.smithi028.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T09:20:30.181 INFO:teuthology.orchestra.run.smithi028.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-19T09:20:30.228 INFO:teuthology.orchestra.run.smithi028.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-09-19T09:20:30.251 INFO:teuthology.orchestra.run.smithi028.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-19T09:20:30.276 INFO:teuthology.orchestra.run.smithi028.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.2 MB/s | 215 kB 00:00 2024-09-19T09:20:30.312 INFO:teuthology.orchestra.run.smithi038.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-19T09:20:30.396 INFO:teuthology.orchestra.run.smithi038.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-19T09:20:30.409 INFO:teuthology.orchestra.run.smithi028.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-09-19T09:20:30.421 INFO:teuthology.orchestra.run.smithi038.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-19T09:20:30.460 INFO:teuthology.orchestra.run.smithi028.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-19T09:20:30.527 INFO:teuthology.orchestra.run.smithi028.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-19T09:20:30.560 INFO:teuthology.orchestra.run.smithi028.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi028.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-09-19T09:20:30.611 INFO:teuthology.orchestra.run.smithi028.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-09-19T09:20:30.636 INFO:teuthology.orchestra.run.smithi028.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-19T09:20:30.655 INFO:teuthology.orchestra.run.smithi038.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-19T09:20:30.661 INFO:teuthology.orchestra.run.smithi028.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T09:20:30.686 INFO:teuthology.orchestra.run.smithi028.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T09:20:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-09-19T09:20:30.712 INFO:teuthology.orchestra.run.smithi028.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-19T09:20:30.713 INFO:teuthology.orchestra.run.smithi038.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T09:20:30.764 INFO:teuthology.orchestra.run.smithi038.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi028.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-19T09:20:30.847 INFO:teuthology.orchestra.run.smithi038.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-19T09:20:30.854 INFO:teuthology.orchestra.run.smithi028.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-19T09:20:30.879 INFO:teuthology.orchestra.run.smithi028.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-19T09:20:30.904 INFO:teuthology.orchestra.run.smithi028.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T09:20:30.931 INFO:teuthology.orchestra.run.smithi038.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T09:20:30.963 INFO:teuthology.orchestra.run.smithi028.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-19T09:20:30.981 INFO:teuthology.orchestra.run.smithi038.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-19T09:20:30.988 INFO:teuthology.orchestra.run.smithi028.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-19T09:20:31.014 INFO:teuthology.orchestra.run.smithi028.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-19T09:20:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T09:20:31.064 INFO:teuthology.orchestra.run.smithi028.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-09-19T09:20:31.090 INFO:teuthology.orchestra.run.smithi038.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T09:20:31.149 INFO:teuthology.orchestra.run.smithi038.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-09-19T09:20:31.331 INFO:teuthology.orchestra.run.smithi028.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.0 MB/s | 1.6 MB 00:00 2024-09-19T09:20:31.332 INFO:teuthology.orchestra.run.smithi038.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 6.2 MB/s | 5.8 MB 00:00 2024-09-19T09:20:31.416 INFO:teuthology.orchestra.run.smithi038.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-19T09:20:31.474 INFO:teuthology.orchestra.run.smithi038.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T09:20:31.497 INFO:teuthology.orchestra.run.smithi028.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-19T09:20:31.542 INFO:teuthology.orchestra.run.smithi038.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-19T09:20:31.553 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-19T09:20:31.556 INFO:teuthology.orchestra.run.smithi028.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T09:20:31.592 INFO:teuthology.orchestra.run.smithi038.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-09-19T09:20:31.675 INFO:teuthology.orchestra.run.smithi038.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-19T09:20:31.756 INFO:teuthology.orchestra.run.smithi028.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-09-19T09:20:31.776 INFO:teuthology.orchestra.run.smithi038.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-19T09:20:31.801 INFO:teuthology.orchestra.run.smithi038.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-19T09:20:31.843 INFO:teuthology.orchestra.run.smithi038.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-19T09:20:31.868 INFO:teuthology.orchestra.run.smithi038.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-19T09:20:31.873 INFO:teuthology.orchestra.run.smithi028.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.6 MB/s | 4.9 MB 00:00 2024-09-19T09:20:31.893 INFO:teuthology.orchestra.run.smithi038.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T09:20:31.932 INFO:teuthology.orchestra.run.smithi028.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T09:20:31.944 INFO:teuthology.orchestra.run.smithi038.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-19T09:20:31.990 INFO:teuthology.orchestra.run.smithi028.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-19T09:20:32.002 INFO:teuthology.orchestra.run.smithi038.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-19T09:20:32.033 INFO:teuthology.orchestra.run.smithi028.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T09:20:32.061 INFO:teuthology.orchestra.run.smithi038.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T09:20:32.116 INFO:teuthology.orchestra.run.smithi028.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-19T09:20:32.128 INFO:teuthology.orchestra.run.smithi038.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.2 MB/s | 5.1 MB 00:00 2024-09-19T09:20:32.145 INFO:teuthology.orchestra.run.smithi038.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 429 kB/s | 36 kB 00:00 2024-09-19T09:20:32.170 INFO:teuthology.orchestra.run.smithi038.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T09:20:32.199 INFO:teuthology.orchestra.run.smithi028.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T09:20:32.212 INFO:teuthology.orchestra.run.smithi038.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T09:20:32.237 INFO:teuthology.orchestra.run.smithi038.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-19T09:20:32.241 INFO:teuthology.orchestra.run.smithi028.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-09-19T09:20:32.271 INFO:teuthology.orchestra.run.smithi038.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T09:20:32.296 INFO:teuthology.orchestra.run.smithi038.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T09:20:32.313 INFO:teuthology.orchestra.run.smithi038.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-19T09:20:32.316 INFO:teuthology.orchestra.run.smithi028.stdout:(78/156): postgresql-server-13.16-1.el9.x86_64. 7.6 MB/s | 5.8 MB 00:00 2024-09-19T09:20:32.342 INFO:teuthology.orchestra.run.smithi028.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-19T09:20:32.347 INFO:teuthology.orchestra.run.smithi038.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-19T09:20:32.349 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-19T09:20:32.367 INFO:teuthology.orchestra.run.smithi028.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-19T09:20:32.393 INFO:teuthology.orchestra.run.smithi028.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-19T09:20:32.397 INFO:teuthology.orchestra.run.smithi038.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T09:20:32.431 INFO:teuthology.orchestra.run.smithi038.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T09:20:32.456 INFO:teuthology.orchestra.run.smithi038.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-19T09:20:32.476 INFO:teuthology.orchestra.run.smithi028.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-19T09:20:32.499 INFO:teuthology.orchestra.run.smithi038.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-09-19T09:20:32.535 INFO:teuthology.orchestra.run.smithi028.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T09:20:32.549 INFO:teuthology.orchestra.run.smithi038.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-19T09:20:32.582 INFO:teuthology.orchestra.run.smithi038.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T09:20:32.585 INFO:teuthology.orchestra.run.smithi028.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T09:20:32.616 INFO:teuthology.orchestra.run.smithi038.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T09:20:32.649 INFO:teuthology.orchestra.run.smithi038.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T09:20:32.664 INFO:teuthology.orchestra.run.smithi028.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 535 kB/s | 42 kB 00:00 2024-09-19T09:20:32.716 INFO:teuthology.orchestra.run.smithi038.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-19T09:20:32.750 INFO:teuthology.orchestra.run.smithi038.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-19T09:20:32.752 INFO:teuthology.orchestra.run.smithi028.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.0 MB/s | 269 kB 00:00 2024-09-19T09:20:32.775 INFO:teuthology.orchestra.run.smithi038.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T09:20:32.801 INFO:teuthology.orchestra.run.smithi038.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-19T09:20:32.811 INFO:teuthology.orchestra.run.smithi028.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T09:20:32.826 INFO:teuthology.orchestra.run.smithi038.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-19T09:20:32.851 INFO:teuthology.orchestra.run.smithi038.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-19T09:20:32.877 INFO:teuthology.orchestra.run.smithi038.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T09:20:32.894 INFO:teuthology.orchestra.run.smithi028.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-19T09:20:32.910 INFO:teuthology.orchestra.run.smithi038.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T09:20:32.935 INFO:teuthology.orchestra.run.smithi038.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T09:20:32.961 INFO:teuthology.orchestra.run.smithi038.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T09:20:32.986 INFO:teuthology.orchestra.run.smithi038.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T09:20:32.995 INFO:teuthology.orchestra.run.smithi028.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-09-19T09:20:33.011 INFO:teuthology.orchestra.run.smithi038.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-19T09:20:33.020 INFO:teuthology.orchestra.run.smithi028.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 407 kB/s | 51 kB 00:00 2024-09-19T09:20:33.037 INFO:teuthology.orchestra.run.smithi038.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-19T09:20:33.062 INFO:teuthology.orchestra.run.smithi038.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-19T09:20:33.087 INFO:teuthology.orchestra.run.smithi028.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.1 MB/s | 5.1 MB 00:00 2024-09-19T09:20:33.088 INFO:teuthology.orchestra.run.smithi038.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-19T09:20:33.112 INFO:teuthology.orchestra.run.smithi028.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-09-19T09:20:33.112 INFO:teuthology.orchestra.run.smithi038.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-19T09:20:33.137 INFO:teuthology.orchestra.run.smithi028.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-19T09:20:33.163 INFO:teuthology.orchestra.run.smithi028.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-19T09:20:33.171 INFO:teuthology.orchestra.run.smithi038.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T09:20:33.188 INFO:teuthology.orchestra.run.smithi028.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T09:20:33.196 INFO:teuthology.orchestra.run.smithi038.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T09:20:33.213 INFO:teuthology.orchestra.run.smithi028.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-19T09:20:33.222 INFO:teuthology.orchestra.run.smithi038.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T09:20:33.238 INFO:teuthology.orchestra.run.smithi028.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-19T09:20:33.255 INFO:teuthology.orchestra.run.smithi038.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T09:20:33.281 INFO:teuthology.orchestra.run.smithi038.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-19T09:20:33.289 INFO:teuthology.orchestra.run.smithi028.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T09:20:33.306 INFO:teuthology.orchestra.run.smithi038.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-19T09:20:33.314 INFO:teuthology.orchestra.run.smithi028.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-19T09:20:33.331 INFO:teuthology.orchestra.run.smithi038.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T09:20:33.339 INFO:teuthology.orchestra.run.smithi028.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T09:20:33.356 INFO:teuthology.orchestra.run.smithi038.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T09:20:33.364 INFO:teuthology.orchestra.run.smithi028.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T09:20:33.377 INFO:teuthology.orchestra.run.smithi038.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-19T09:20:33.389 INFO:teuthology.orchestra.run.smithi028.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-19T09:20:33.407 INFO:teuthology.orchestra.run.smithi038.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-09-19T09:20:33.432 INFO:teuthology.orchestra.run.smithi038.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T09:20:33.440 INFO:teuthology.orchestra.run.smithi028.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-19T09:20:33.458 INFO:teuthology.orchestra.run.smithi038.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T09:20:33.483 INFO:teuthology.orchestra.run.smithi038.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-19T09:20:33.490 INFO:teuthology.orchestra.run.smithi028.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T09:20:33.508 INFO:teuthology.orchestra.run.smithi038.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T09:20:33.515 INFO:teuthology.orchestra.run.smithi028.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-19T09:20:33.534 INFO:teuthology.orchestra.run.smithi038.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T09:20:33.540 INFO:teuthology.orchestra.run.smithi028.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-19T09:20:33.559 INFO:teuthology.orchestra.run.smithi038.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T09:20:33.576 INFO:teuthology.orchestra.run.smithi038.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T09:20:33.582 INFO:teuthology.orchestra.run.smithi028.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-09-19T09:20:33.601 INFO:teuthology.orchestra.run.smithi038.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-19T09:20:33.633 INFO:teuthology.orchestra.run.smithi028.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-19T09:20:33.657 INFO:teuthology.orchestra.run.smithi038.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-09-19T09:20:33.666 INFO:teuthology.orchestra.run.smithi028.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T09:20:33.691 INFO:teuthology.orchestra.run.smithi028.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-19T09:20:33.710 INFO:teuthology.orchestra.run.smithi038.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T09:20:33.725 INFO:teuthology.orchestra.run.smithi028.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T09:20:33.801 INFO:teuthology.orchestra.run.smithi028.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-19T09:20:33.826 INFO:teuthology.orchestra.run.smithi028.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-19T09:20:33.851 INFO:teuthology.orchestra.run.smithi028.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T09:20:33.876 INFO:teuthology.orchestra.run.smithi028.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T09:20:33.902 INFO:teuthology.orchestra.run.smithi028.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-19T09:20:33.927 INFO:teuthology.orchestra.run.smithi028.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-19T09:20:33.952 INFO:teuthology.orchestra.run.smithi028.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-19T09:20:33.961 INFO:teuthology.orchestra.run.smithi038.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T09:20:33.986 INFO:teuthology.orchestra.run.smithi028.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T09:20:34.011 INFO:teuthology.orchestra.run.smithi028.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T09:20:34.019 INFO:teuthology.orchestra.run.smithi038.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T09:20:34.045 INFO:teuthology.orchestra.run.smithi028.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T09:20:34.078 INFO:teuthology.orchestra.run.smithi028.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T09:20:34.104 INFO:teuthology.orchestra.run.smithi028.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi028.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-19T09:20:34.154 INFO:teuthology.orchestra.run.smithi028.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-19T09:20:34.171 INFO:teuthology.orchestra.run.smithi028.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-19T09:20:34.188 INFO:teuthology.orchestra.run.smithi028.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-19T09:20:34.246 INFO:teuthology.orchestra.run.smithi028.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T09:20:34.264 INFO:teuthology.orchestra.run.smithi028.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-19T09:20:34.305 INFO:teuthology.orchestra.run.smithi028.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T09:20:34.333 INFO:teuthology.orchestra.run.smithi028.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-19T09:20:34.348 INFO:teuthology.orchestra.run.smithi028.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-19T09:20:34.364 INFO:teuthology.orchestra.run.smithi028.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-19T09:20:34.369 INFO:teuthology.orchestra.run.smithi038.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2024-09-19T09:20:34.390 INFO:teuthology.orchestra.run.smithi028.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-19T09:20:34.411 INFO:teuthology.orchestra.run.smithi038.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.6 MB/s | 1.8 MB 00:00 2024-09-19T09:20:34.415 INFO:teuthology.orchestra.run.smithi028.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T09:20:34.437 INFO:teuthology.orchestra.run.smithi038.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 612 kB/s | 41 kB 00:00 2024-09-19T09:20:34.440 INFO:teuthology.orchestra.run.smithi028.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-19T09:20:34.465 INFO:teuthology.orchestra.run.smithi028.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T09:20:34.491 INFO:teuthology.orchestra.run.smithi028.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T09:20:34.495 INFO:teuthology.orchestra.run.smithi038.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T09:20:34.516 INFO:teuthology.orchestra.run.smithi028.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T09:20:34.541 INFO:teuthology.orchestra.run.smithi028.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T09:20:34.566 INFO:teuthology.orchestra.run.smithi028.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T09:20:34.592 INFO:teuthology.orchestra.run.smithi028.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T09:20:34.617 INFO:teuthology.orchestra.run.smithi028.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T09:20:34.620 INFO:teuthology.orchestra.run.smithi038.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 1.4 MB 00:00 2024-09-19T09:20:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-19T09:20:34.668 INFO:teuthology.orchestra.run.smithi028.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-19T09:20:34.693 INFO:teuthology.orchestra.run.smithi028.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-19T09:20:34.752 INFO:teuthology.orchestra.run.smithi028.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-19T09:20:34.837 INFO:teuthology.orchestra.run.smithi038.stdout:(155/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-09-19T09:20:34.938 INFO:teuthology.orchestra.run.smithi038.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 2.4 MB 00:00 2024-09-19T09:20:34.943 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:20:34.943 INFO:teuthology.orchestra.run.smithi038.stdout:Total 19 MB/s | 226 MB 00:12 2024-09-19T09:20:35.027 INFO:teuthology.orchestra.run.smithi028.stdout:(148/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.1 MB/s | 16 MB 00:02 2024-09-19T09:20:35.102 INFO:teuthology.orchestra.run.smithi028.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-19T09:20:35.203 INFO:teuthology.orchestra.run.smithi028.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 6.7 MB/s | 3.4 MB 00:00 2024-09-19T09:20:35.253 INFO:teuthology.orchestra.run.smithi028.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-09-19T09:20:35.286 INFO:teuthology.orchestra.run.smithi028.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 9.8 MB/s | 1.8 MB 00:00 2024-09-19T09:20:35.311 INFO:teuthology.orchestra.run.smithi028.stdout:(153/156): NetworkManager-team-1.51.0-1.el9.x86 377 kB/s | 41 kB 00:00 2024-09-19T09:20:35.337 INFO:teuthology.orchestra.run.smithi028.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T09:20:35.370 INFO:teuthology.orchestra.run.smithi028.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-19T09:20:35.445 INFO:teuthology.orchestra.run.smithi028.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-19T09:20:35.450 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:20:35.451 INFO:teuthology.orchestra.run.smithi028.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-19T09:20:36.194 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-19T09:20:36.362 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-19T09:20:36.362 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-19T09:20:36.723 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T09:20:36.893 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T09:20:36.893 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T09:20:37.772 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-19T09:20:38.277 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-19T09:20:38.277 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-19T09:20:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T09:20:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T09:20:39.907 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:39.908 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:20:39.919 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:20:39.920 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:20:39.921 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:20:39.922 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:20:39.923 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:20:39.926 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:20:39.926 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:20:39.989 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:20:40.004 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:20:40.005 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:20:40.006 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:20:40.007 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:20:40.008 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:20:40.009 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:20:40.010 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:20:40.011 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:20:40.012 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:20:40.013 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:20:40.014 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:20:40.015 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:20:40.016 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:20:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:20:40.018 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:20:40.019 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:20:40.020 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-19T09:20:40.021 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-09-19T09:20:40.022 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 8 Packages 2024-09-19T09:20:40.022 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:20:40.022 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 226 M 2024-09-19T09:20:40.022 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-09-19T09:20:40.590 INFO:teuthology.orchestra.run.smithi078.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T09:20:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.6 MB/s | 896 kB 00:00 2024-09-19T09:20:40.857 INFO:teuthology.orchestra.run.smithi078.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T09:20:40.933 INFO:teuthology.orchestra.run.smithi078.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T09:20:41.066 INFO:teuthology.orchestra.run.smithi078.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-19T09:20:41.125 INFO:teuthology.orchestra.run.smithi078.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 8.9 MB/s | 2.4 MB 00:00 2024-09-19T09:20:41.342 INFO:teuthology.orchestra.run.smithi078.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T09:20:41.642 INFO:teuthology.orchestra.run.smithi078.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T09:20:41.676 INFO:teuthology.orchestra.run.smithi078.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 764 kB/s | 25 kB 00:00 2024-09-19T09:20:41.959 INFO:teuthology.orchestra.run.smithi078.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T09:20:42.010 INFO:teuthology.orchestra.run.smithi078.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T09:20:42.210 INFO:teuthology.orchestra.run.smithi078.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 16 MB/s | 17 MB 00:01 2024-09-19T09:20:42.227 INFO:teuthology.orchestra.run.smithi078.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 125 kB/s | 27 kB 00:00 2024-09-19T09:20:42.244 INFO:teuthology.orchestra.run.smithi078.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 956 kB/s | 32 kB 00:00 2024-09-19T09:20:42.286 INFO:teuthology.orchestra.run.smithi078.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T09:20:42.320 INFO:teuthology.orchestra.run.smithi078.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T09:20:42.345 INFO:teuthology.orchestra.run.smithi078.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T09:20:42.378 INFO:teuthology.orchestra.run.smithi078.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.5 MB/s | 506 kB 00:00 2024-09-19T09:20:42.445 INFO:teuthology.orchestra.run.smithi078.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 676 kB/s | 45 kB 00:00 2024-09-19T09:20:42.588 INFO:teuthology.orchestra.run.smithi078.stdout:(20/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 22 MB/s | 5.3 MB 00:00 2024-09-19T09:20:42.613 INFO:teuthology.orchestra.run.smithi078.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 816 kB/s | 136 kB 00:00 2024-09-19T09:20:42.638 INFO:teuthology.orchestra.run.smithi078.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T09:20:42.663 INFO:teuthology.orchestra.run.smithi078.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T09:20:42.697 INFO:teuthology.orchestra.run.smithi078.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T09:20:42.722 INFO:teuthology.orchestra.run.smithi078.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T09:20:42.747 INFO:teuthology.orchestra.run.smithi078.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T09:20:42.798 INFO:teuthology.orchestra.run.smithi078.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T09:20:42.873 INFO:teuthology.orchestra.run.smithi078.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T09:20:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 316 kB/s | 32 kB 00:00 2024-09-19T09:20:42.925 INFO:teuthology.orchestra.run.smithi078.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T09:20:43.308 INFO:teuthology.orchestra.run.smithi078.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 19 MB/s | 7.4 MB 00:00 2024-09-19T09:20:43.375 INFO:teuthology.orchestra.run.smithi078.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T09:20:43.425 INFO:teuthology.orchestra.run.smithi078.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 988 kB/s | 49 kB 00:00 2024-09-19T09:20:43.500 INFO:teuthology.orchestra.run.smithi078.stdout:(34/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 5.9 MB/s | 3.6 MB 00:00 2024-09-19T09:20:43.517 INFO:teuthology.orchestra.run.smithi078.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 179 kB/s | 16 kB 00:00 2024-09-19T09:20:43.951 INFO:teuthology.orchestra.run.smithi078.stdout:(36/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 21 MB/s | 49 MB 00:02 2024-09-19T09:20:43.985 INFO:teuthology.orchestra.run.smithi078.stdout:(37/156): ceph-volume-19.1.1-361.g2f599a46.el9. 589 kB/s | 285 kB 00:00 2024-09-19T09:20:44.001 INFO:teuthology.orchestra.run.smithi078.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 367 kB/s | 18 kB 00:00 2024-09-19T09:20:44.035 INFO:teuthology.orchestra.run.smithi078.stdout:(39/156): cephadm-19.1.1-361.g2f599a46.el9.noar 1.4 MB/s | 765 kB 00:00 2024-09-19T09:20:44.085 INFO:teuthology.orchestra.run.smithi078.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 793 kB/s | 80 kB 00:00 2024-09-19T09:20:44.136 INFO:teuthology.orchestra.run.smithi078.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-19T09:20:44.186 INFO:teuthology.orchestra.run.smithi078.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-19T09:20:44.211 INFO:teuthology.orchestra.run.smithi078.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-19T09:20:44.278 INFO:teuthology.orchestra.run.smithi078.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-19T09:20:44.354 INFO:teuthology.orchestra.run.smithi078.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-19T09:20:44.379 INFO:teuthology.orchestra.run.smithi078.stdout:(46/156): python3-ply-3.11-14.el9.noarch.rpm 554 kB/s | 106 kB 00:00 2024-09-19T09:20:44.412 INFO:teuthology.orchestra.run.smithi078.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-19T09:20:44.463 INFO:teuthology.orchestra.run.smithi078.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-19T09:20:44.630 INFO:teuthology.orchestra.run.smithi078.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-19T09:20:44.672 INFO:teuthology.orchestra.run.smithi078.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-19T09:20:44.705 INFO:teuthology.orchestra.run.smithi078.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-19T09:20:44.756 INFO:teuthology.orchestra.run.smithi078.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-19T09:20:44.856 INFO:teuthology.orchestra.run.smithi078.stdout:(53/156): elfutils-libelf-devel-0.191-4.el9.x86 96 kB/s | 22 kB 00:00 2024-09-19T09:20:44.881 INFO:teuthology.orchestra.run.smithi078.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-09-19T09:20:44.931 INFO:teuthology.orchestra.run.smithi078.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-09-19T09:20:44.957 INFO:teuthology.orchestra.run.smithi078.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-19T09:20:44.982 INFO:teuthology.orchestra.run.smithi078.stdout:(57/156): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-19T09:20:45.007 INFO:teuthology.orchestra.run.smithi078.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-19T09:20:45.016 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-19T09:20:45.025 INFO:teuthology.orchestra.run.smithi078.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-19T09:20:45.066 INFO:teuthology.orchestra.run.smithi078.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-19T09:20:45.083 INFO:teuthology.orchestra.run.smithi078.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T09:20:45.108 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:20:45.117 INFO:teuthology.orchestra.run.smithi078.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-19T09:20:45.147 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:20:45.150 INFO:teuthology.orchestra.run.smithi078.stdout:(63/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.8 MB/s | 662 kB 00:00 2024-09-19T09:20:45.175 INFO:teuthology.orchestra.run.smithi078.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-19T09:20:45.201 INFO:teuthology.orchestra.run.smithi078.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T09:20:45.201 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:20:45.218 INFO:teuthology.orchestra.run.smithi078.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 573 kB/s | 38 kB 00:00 2024-09-19T09:20:45.243 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:20:45.276 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:20:45.320 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:20:45.401 INFO:teuthology.orchestra.run.smithi078.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 22 MB/s | 4.9 MB 00:00 2024-09-19T09:20:45.477 INFO:teuthology.orchestra.run.smithi078.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-09-19T09:20:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-19T09:20:45.581 INFO:teuthology.orchestra.run.smithi078.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-19T09:20:45.603 INFO:teuthology.orchestra.run.smithi078.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-19T09:20:45.653 INFO:teuthology.orchestra.run.smithi078.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-19T09:20:45.676 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:20:45.686 INFO:teuthology.orchestra.run.smithi078.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-19T09:20:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T09:20:45.770 INFO:teuthology.orchestra.run.smithi078.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T09:20:45.785 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:45.787 INFO:teuthology.orchestra.run.smithi078.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-19T09:20:45.808 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:20:45.829 INFO:teuthology.orchestra.run.smithi078.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T09:20:45.854 INFO:teuthology.orchestra.run.smithi078.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-19T09:20:45.880 INFO:teuthology.orchestra.run.smithi078.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-19T09:20:45.901 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:20:45.905 INFO:teuthology.orchestra.run.smithi078.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T09:20:45.910 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:45.936 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:45.938 INFO:teuthology.orchestra.run.smithi078.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T09:20:45.964 INFO:teuthology.orchestra.run.smithi078.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-19T09:20:46.022 INFO:teuthology.orchestra.run.smithi078.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-19T09:20:46.028 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:46.057 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:20:46.088 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:20:46.089 INFO:teuthology.orchestra.run.smithi078.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-19T09:20:46.125 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:20:46.150 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:20:46.156 INFO:teuthology.orchestra.run.smithi078.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-19T09:20:46.182 INFO:teuthology.orchestra.run.smithi078.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 839 kB/s | 77 kB 00:00 2024-09-19T09:20:46.204 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:20:46.207 INFO:teuthology.orchestra.run.smithi078.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-19T09:20:46.221 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:20:46.241 INFO:teuthology.orchestra.run.smithi078.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T09:20:46.266 INFO:teuthology.orchestra.run.smithi078.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-19T09:20:46.279 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:20:46.315 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:46.341 INFO:teuthology.orchestra.run.smithi078.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 8.7 MB/s | 6.0 MB 00:00 2024-09-19T09:20:46.366 INFO:teuthology.orchestra.run.smithi078.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-19T09:20:46.400 INFO:teuthology.orchestra.run.smithi078.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-09-19T09:20:46.421 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:46.425 INFO:teuthology.orchestra.run.smithi078.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-09-19T09:20:46.450 INFO:teuthology.orchestra.run.smithi078.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-19T09:20:46.458 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:46.484 INFO:teuthology.orchestra.run.smithi078.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-09-19T09:20:46.509 INFO:teuthology.orchestra.run.smithi078.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T09:20:46.520 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:46.543 INFO:teuthology.orchestra.run.smithi078.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-19T09:20:46.551 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:46.553 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:20:46.568 INFO:teuthology.orchestra.run.smithi078.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-19T09:20:46.571 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:46.627 INFO:teuthology.orchestra.run.smithi078.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.5 MB/s | 304 kB 00:00 2024-09-19T09:20:46.653 INFO:teuthology.orchestra.run.smithi078.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 688 kB/s | 58 kB 00:00 2024-09-19T09:20:46.686 INFO:teuthology.orchestra.run.smithi078.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T09:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:46.711 INFO:teuthology.orchestra.run.smithi078.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T09:20:46.753 INFO:teuthology.orchestra.run.smithi078.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-09-19T09:20:46.760 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:20:46.808 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:46.842 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:46.897 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:20:46.929 INFO:teuthology.orchestra.run.smithi078.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-19T09:20:46.941 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:46.954 INFO:teuthology.orchestra.run.smithi078.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-09-19T09:20:46.972 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:20:47.004 INFO:teuthology.orchestra.run.smithi078.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-19T09:20:47.007 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:20:47.030 INFO:teuthology.orchestra.run.smithi078.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-19T09:20:47.063 INFO:teuthology.orchestra.run.smithi078.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-19T09:20:47.088 INFO:teuthology.orchestra.run.smithi078.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T09:20:47.110 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:47.113 INFO:teuthology.orchestra.run.smithi078.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T09:20:47.147 INFO:teuthology.orchestra.run.smithi078.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T09:20:47.150 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:20:47.181 INFO:teuthology.orchestra.run.smithi078.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-19T09:20:47.183 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:47.202 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:20:47.206 INFO:teuthology.orchestra.run.smithi078.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-19T09:20:47.229 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:47.265 INFO:teuthology.orchestra.run.smithi078.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T09:20:47.290 INFO:teuthology.orchestra.run.smithi078.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T09:20:47.315 INFO:teuthology.orchestra.run.smithi078.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T09:20:47.336 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:47.340 INFO:teuthology.orchestra.run.smithi078.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-19T09:20:47.363 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:47.374 INFO:teuthology.orchestra.run.smithi078.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-09-19T09:20:47.408 INFO:teuthology.orchestra.run.smithi078.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-19T09:20:47.410 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:47.426 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:47.441 INFO:teuthology.orchestra.run.smithi078.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T09:20:47.447 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:47.467 INFO:teuthology.orchestra.run.smithi078.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T09:20:47.470 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:47.529 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:20:47.534 INFO:teuthology.orchestra.run.smithi078.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-19T09:20:47.559 INFO:teuthology.orchestra.run.smithi078.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-19T09:20:47.584 INFO:teuthology.orchestra.run.smithi078.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-19T09:20:47.609 INFO:teuthology.orchestra.run.smithi078.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-19T09:20:47.630 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:20:47.634 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:20:47.635 INFO:teuthology.orchestra.run.smithi078.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T09:20:47.660 INFO:teuthology.orchestra.run.smithi078.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-09-19T09:20:47.681 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:20:47.686 INFO:teuthology.orchestra.run.smithi078.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 384 kB/s | 19 kB 00:00 2024-09-19T09:20:47.711 INFO:teuthology.orchestra.run.smithi078.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-09-19T09:20:47.715 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:20:47.744 INFO:teuthology.orchestra.run.smithi078.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T09:20:47.778 INFO:teuthology.orchestra.run.smithi078.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T09:20:47.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:20:47.803 INFO:teuthology.orchestra.run.smithi078.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T09:20:47.830 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:20:47.845 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:20:47.885 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:20:47.895 INFO:teuthology.orchestra.run.smithi078.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-19T09:20:47.929 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:20:47.929 INFO:teuthology.orchestra.run.smithi078.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-19T09:20:47.962 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:20:47.969 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:20:47.971 INFO:teuthology.orchestra.run.smithi078.stdout:(134/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-19T09:20:47.996 INFO:teuthology.orchestra.run.smithi078.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-09-19T09:20:47.997 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:20:48.022 INFO:teuthology.orchestra.run.smithi078.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T09:20:48.047 INFO:teuthology.orchestra.run.smithi078.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T09:20:48.069 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:20:48.072 INFO:teuthology.orchestra.run.smithi078.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 631 kB/s | 90 kB 00:00 2024-09-19T09:20:48.091 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:48.097 INFO:teuthology.orchestra.run.smithi078.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T09:20:48.122 INFO:teuthology.orchestra.run.smithi078.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T09:20:48.129 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:20:48.148 INFO:teuthology.orchestra.run.smithi078.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T09:20:48.158 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:20:48.172 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:48.173 INFO:teuthology.orchestra.run.smithi078.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-19T09:20:48.194 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:20:48.198 INFO:teuthology.orchestra.run.smithi078.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T09:20:48.224 INFO:teuthology.orchestra.run.smithi078.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-19T09:20:48.249 INFO:teuthology.orchestra.run.smithi078.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T09:20:48.272 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:20:48.274 INFO:teuthology.orchestra.run.smithi078.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T09:20:48.299 INFO:teuthology.orchestra.run.smithi078.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T09:20:48.313 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:20:48.341 INFO:teuthology.orchestra.run.smithi078.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-19T09:20:48.365 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:20:48.402 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:20:48.407 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:20:48.475 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:20:48.524 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:20:48.525 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:20:48.525 INFO:teuthology.orchestra.run.smithi078.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-19T09:20:48.567 INFO:teuthology.orchestra.run.smithi078.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-09-19T09:20:48.603 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:20:48.619 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:20:48.626 INFO:teuthology.orchestra.run.smithi078.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-19T09:20:48.652 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:20:48.668 INFO:teuthology.orchestra.run.smithi078.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 408 kB/s | 41 kB 00:00 2024-09-19T09:20:48.697 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:20:48.726 INFO:teuthology.orchestra.run.smithi078.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 9.0 MB/s | 1.8 MB 00:00 2024-09-19T09:20:48.749 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:20:48.751 INFO:teuthology.orchestra.run.smithi078.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-19T09:20:48.771 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:20:48.813 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:20:48.822 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:20:48.843 INFO:teuthology.orchestra.run.smithi078.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-19T09:20:48.870 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:20:48.918 INFO:teuthology.orchestra.run.smithi078.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-19T09:20:48.922 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:20:48.924 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:20:48.925 INFO:teuthology.orchestra.run.smithi078.stdout:Total 25 MB/s | 226 MB 00:08 2024-09-19T09:20:48.954 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:20:48.981 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:20:49.054 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:20:49.105 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:20:49.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:20:49.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:20:49.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:20:49.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:20:49.326 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:20:49.352 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:20:49.394 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:20:49.425 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:20:49.475 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:20:49.484 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:20:49.527 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:20:49.553 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:20:49.569 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:20:49.607 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:20:49.658 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:20:49.700 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:20:49.754 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:20:49.831 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:20:49.886 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:20:49.952 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:20:50.033 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:20:50.079 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:20:50.105 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-09-19T09:20:50.115 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:20:50.163 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:20:50.197 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:20:50.241 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:20:50.256 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-09-19T09:20:50.256 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-09-19T09:20:50.284 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:20:50.308 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:20:50.393 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:20:50.427 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:20:50.429 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:20:50.470 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:20:50.470 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:20:50.508 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:20:50.512 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:20:50.551 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:20:50.727 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:20:50.786 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:20:50.842 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:20:50.857 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:20:50.911 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:20:50.916 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:20:50.951 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:20:50.953 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:20:50.976 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:20:50.989 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:20:51.001 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:20:51.037 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:20:51.081 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:20:51.161 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:20:51.208 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:20:51.259 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:20:51.287 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:20:51.321 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:20:51.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:20:51.400 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:20:51.400 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:20:51.430 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:20:51.442 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:20:51.464 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:20:51.498 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:20:51.540 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:20:51.582 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:20:51.589 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:20:51.633 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:20:51.675 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:20:51.681 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:20:51.730 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:20:51.761 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:20:51.782 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:20:51.863 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:20:51.863 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:20:51.869 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:20:51.890 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:20:51.919 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:20:51.939 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:20:52.010 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:20:52.049 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-09-19T09:20:52.050 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-09-19T09:20:52.146 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:20:52.180 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:20:52.221 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:20:52.267 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:20:52.306 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:20:52.346 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:20:52.376 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:20:52.410 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:20:52.444 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:20:52.486 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:20:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:20:52.610 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:20:52.653 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:20:52.684 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:20:52.720 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:20:52.767 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:20:52.801 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:20:52.845 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:20:52.909 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:20:52.965 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:20:53.019 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:20:53.110 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:20:53.145 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:20:53.340 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:20:53.376 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:20:53.457 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:20:53.612 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:20:53.657 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:20:53.690 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:20:53.794 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:20:53.854 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:20:53.909 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:20:54.000 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:20:54.043 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:20:54.154 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:20:54.234 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:20:54.264 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:20:54.475 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:20:54.608 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:20:55.310 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:20:55.434 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:20:55.592 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:20:55.700 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:20:55.733 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:20:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:20:56.103 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:20:56.187 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:20:56.217 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:20:56.254 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:20:56.763 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:20:56.783 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:20:56.798 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:20:56.857 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:20:56.870 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:20:56.885 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:20:56.900 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:20:56.918 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:20:57.248 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:20:57.297 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:20:57.398 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:20:57.427 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:20:57.464 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:20:57.977 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:20:58.008 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:20:58.064 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:20:58.093 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:20:58.263 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-09-19T09:20:58.389 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:20:58.427 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:20:58.480 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:20:58.523 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:20:58.549 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:20:58.591 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:20:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:20:59.024 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:20:59.192 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:59.283 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:20:59.306 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:20:59.322 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:20:59.346 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:20:59.477 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:20:59.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:20:59.563 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:20:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:20:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:59.807 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:20:59.923 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:20:59.958 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:00.000 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:21:00.052 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:00.052 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:21:00.053 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:21:00.053 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:00.084 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:00.133 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:21:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:21:00.360 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:21:00.423 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:21:00.465 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:00.498 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:00.649 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:21:00.758 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:21:00.876 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:21:00.917 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:21:00.959 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:21:01.041 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:21:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:01.093 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:21:01.125 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:21:01.126 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:01.169 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:21:01.201 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:21:01.229 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:21:01.229 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:01.230 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:21:01.230 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:21:01.230 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:01.269 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:01.306 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:21:01.368 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:21:01.398 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:21:01.442 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:21:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:21:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:21:01.596 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:21:01.625 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:21:01.703 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:21:01.748 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:21:01.830 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:21:01.970 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:21:02.025 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:21:02.543 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:21:02.628 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:21:02.658 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:21:02.693 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:21:02.752 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:21:02.806 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:21:02.968 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:21:03.026 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:21:03.080 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:21:03.162 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:21:03.202 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:21:03.238 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:21:03.285 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:21:03.321 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:21:03.364 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:21:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:21:03.542 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:21:03.591 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:21:03.633 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:21:03.743 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:21:03.973 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:21:04.041 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:21:04.066 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:21:04.090 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:21:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:21:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:21:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:21:04.359 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:21:04.393 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:21:04.440 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:21:04.481 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:21:04.510 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:21:04.545 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:21:04.578 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:21:04.619 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:21:04.654 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:21:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:21:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:21:04.824 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:21:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:21:04.991 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:21:05.707 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:21:05.764 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:21:05.797 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:21:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:21:05.977 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:21:06.022 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:21:06.099 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:21:06.167 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:21:06.351 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:21:06.383 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:21:06.460 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:21:07.097 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:21:07.214 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:21:08.419 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:21:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:21:08.840 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:21:08.873 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:21:09.182 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:21:09.244 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:21:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:21:09.541 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:21:09.577 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:21:10.058 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:21:10.097 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:21:10.177 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:21:10.216 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:21:12.382 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:12.382 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-09-19T09:21:12.382 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-09-19T09:21:12.382 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-09-19T09:21:12.383 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-09-19T09:21:12.383 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-09-19T09:21:12.383 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-09-19T09:21:12.383 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-09-19T09:21:12.383 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:12.571 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:12.587 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:12.588 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:12.588 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:21:12.588 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:21:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:21:13.216 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:21:13.216 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:21:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:13.514 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-09-19T09:21:13.515 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:13.697 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:13.714 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:13.714 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:13.714 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:21:13.714 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:14.123 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:14.150 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:14.150 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:14.150 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:21:14.151 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:14.151 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:14.151 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:14.391 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:14.414 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:14.414 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:14.414 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:21:14.415 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:14.415 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:14.415 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:14.875 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:14.898 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:14.898 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:14.898 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:21:14.899 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:14.899 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:14.899 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:15.023 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:21:15.081 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:21:15.164 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:21:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:15.286 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:15.286 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:21:15.287 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:15.287 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:15.377 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:21:15.427 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:21:15.508 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:21:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:15.607 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:21:15.751 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:21:15.794 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:21:15.825 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:21:16.088 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:16.089 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:21:16.214 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:21:16.277 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:21:16.300 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:16.326 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:16.358 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:21:16.415 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:21:16.449 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:21:16.526 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:21:16.607 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:21:16.697 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:21:16.840 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:21:16.910 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:21:16.950 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:21:16.977 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:21:17.323 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:17.358 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:17.390 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:21:17.455 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:21:17.481 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:17.571 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:17.609 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:18.596 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:18.624 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:21:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:18.995 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:18.995 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:21:18.995 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:18.995 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:21:19.220 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:21:19.648 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:19.675 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:19.786 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:19.955 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:21:19.985 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:20.011 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:20.012 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:20.012 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:21:20.012 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:20.012 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:20.012 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:20.061 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:20.073 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:20.176 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:21:20.199 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:21:20.231 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:20.236 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:20.255 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:20.298 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:20.321 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:20.322 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:20.322 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:21:20.322 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:20.374 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:20.397 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:20.397 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:20.397 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:21:20.398 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:20.398 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:20.398 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:20.716 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:20.740 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:20.740 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:20.740 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:21:20.740 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:20.740 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:20.741 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:21.232 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:21:21.264 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:21.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:21.331 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:21.356 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:21.356 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:21.356 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:21:21.357 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:21.407 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:21.432 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:21.749 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:21.774 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-09-19T09:21:24.910 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-09-19T09:21:24.911 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-09-19T09:21:24.911 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-09-19T09:21:24.911 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:25.087 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:25.103 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:21:25.103 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:25.103 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:21:25.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:25.579 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:21:25.620 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:21:25.662 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:21:25.693 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:21:25.720 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:21:25.923 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:21:25.975 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:21:26.005 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:21:26.035 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:21:26.086 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:21:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:21:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:26.661 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:21:26.705 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:21:26.712 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:26.743 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:21:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:26.775 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:21:26.802 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:21:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:21:26.855 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:26.855 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:21:26.855 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:26.855 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:21:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:27.008 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:21:27.056 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:21:27.086 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:21:27.117 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:21:27.154 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:21:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:21:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:21:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:21:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:21:27.696 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:21:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:21:27.735 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:27.776 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:21:27.792 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:21:27.798 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:27.825 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:21:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:21:27.919 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:21:27.941 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:21:27.961 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:21:27.987 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:21:28.013 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:21:28.013 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:28.089 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:21:28.097 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:28.097 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:28.142 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:21:28.174 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:21:28.181 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:28.201 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:21:28.206 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:21:28.211 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:28.211 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:28.399 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:28.400 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:28.500 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:28.567 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:28.672 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:28.705 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:21:28.708 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:21:28.748 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:21:28.788 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:21:28.818 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:21:28.840 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:21:28.848 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:28.938 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:21:28.966 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:21:28.994 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:21:29.026 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:21:29.026 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:29.110 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:29.110 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:29.185 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:29.210 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:21:29.216 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:29.216 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:29.404 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:29.404 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:29.496 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:29.565 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:29.893 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:21:29.925 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:30.884 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:21:30.909 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:31.011 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:21:31.172 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:21:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:21:31.234 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:31.234 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:21:31.234 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:31.235 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:21:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:31.289 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:21:31.395 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:21:31.451 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:21:32.370 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:21:32.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:21:32.421 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:32.465 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:32.487 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:21:32.488 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:32.488 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:21:32.488 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:32.532 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:21:32.555 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:32.555 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:21:32.555 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:32.555 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:21:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:21:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:21:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:21:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:21:34.203 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:21:34.204 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:21:34.205 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:21:34.217 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:21:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:21:34.219 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:21:34.220 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:21:34.221 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:21:34.222 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:21:34.223 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:21:34.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:21:34.225 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:21:34.226 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:21:34.227 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:21:34.228 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:21:34.229 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:21:34.229 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:21:34.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:21:34.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:21:34.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:21:34.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:21:34.930 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:21:34.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:21:34.932 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:21:34.933 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:21:34.934 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:21:34.935 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:21:34.936 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:21:34.937 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:21:34.938 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:21:34.939 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:21:34.940 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:21:34.941 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:21:36.941 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:36.941 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:21:36.942 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.943 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.944 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:21:36.945 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.946 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:21:36.947 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.948 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:21:36.949 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:21:36.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:21:36.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:21:36.952 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.953 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:21:36.954 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:21:36.957 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:21:36.957 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:21:36.957 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-19T09:21:37.413 DEBUG:teuthology.parallel:result is None 2024-09-19T09:21:37.414 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:21:37.452 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:21:37.501 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:21:37.542 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:21:37.572 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:21:37.750 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:21:37.791 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:21:37.821 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:21:37.851 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:21:37.853 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:37.853 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:37.853 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:21:37.854 INFO:teuthology.orchestra.run.smithi028.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.855 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.856 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.857 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:21:37.858 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:21:37.859 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:21:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:21:37.861 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:37.862 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:37.863 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:21:37.863 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:21:37.863 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:21:37.863 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:21:37.863 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.864 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:21:37.865 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:21:37.866 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:21:37.867 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:21:37.867 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:21:37.867 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:21:37.867 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T09:21:37.889 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:21:38.231 DEBUG:teuthology.parallel:result is None 2024-09-19T09:21:38.543 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:38.579 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:38.601 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:21:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:21:39.517 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:21:39.557 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:21:39.639 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:21:39.712 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:21:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:21:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:21:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:39.863 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:21:39.864 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:39.947 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:21:39.973 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:21:39.978 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:39.978 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:40.158 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:21:40.158 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:40.245 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:21:40.305 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:21:46.254 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:21:46.255 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:21:46.256 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:21:46.268 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:21:46.268 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:21:46.269 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:21:46.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:21:46.271 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:21:46.272 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:21:46.273 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:21:46.274 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:21:46.275 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:21:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:21:46.277 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:21:46.278 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:21:46.279 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:21:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:21:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:21:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:21:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:21:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:21:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:21:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:21:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:21:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:21:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:21:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:21:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:21:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:21:48.705 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-09-19T09:21:49.133 DEBUG:teuthology.parallel:result is None 2024-09-19T09:21:49.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:49.288 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:21:49.317 INFO:teuthology.orchestra.run.smithi028.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.318 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.318 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:21:49.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:49.469 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:21:49.495 INFO:teuthology.orchestra.run.smithi038.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.496 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.496 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:21:49.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:49.639 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:21:49.663 INFO:teuthology.orchestra.run.smithi078.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.663 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:21:49.663 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:21:49.665 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T09:21:49.665 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:21:49.665 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:21:49.699 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:21:49.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:21:49.731 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:21:49.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:21:49.760 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T09:21:49.761 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:21:49.761 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:21:49.790 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:21:49.859 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:21:49.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:21:49.889 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:21:49.958 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:21:49.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:21:49.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:21:50.053 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T09:21:50.053 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:21:50.053 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:21:50.083 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:21:50.150 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:21:50.150 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:21:50.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:21:50.249 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:21:50.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:21:50.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:21:50.343 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T09:21:50.343 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:21:50.343 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:21:50.369 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:21:50.434 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:21:50.434 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:21:50.460 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:21:50.524 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:21:50.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:21:50.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:21:50.618 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T09:21:50.718 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T09:21:50.719 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:50.719 INFO:tasks.cephadm:Cluster fsid is 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:21:50.719 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T09:21:50.720 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.28', 'mon.b': '172.21.15.38', 'mon.c': '172.21.15.78'} 2024-09-19T09:21:50.720 INFO:tasks.cephadm:First mon is mon.a on smithi028 2024-09-19T09:21:50.720 INFO:tasks.cephadm:First mgr is x 2024-09-19T09:21:50.720 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T09:21:50.720 DEBUG:teuthology.orchestra.run.smithi028:> sudo hostname $(hostname -s) 2024-09-19T09:21:50.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-09-19T09:21:50.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-09-19T09:21:50.800 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T09:21:50.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:50.938 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T09:21:51.078 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:21:51.082 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T09:21:51.082 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T09:21:51.083 DEBUG:teuthology.orchestra.run.smithi028:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:51.466 INFO:teuthology.orchestra.run.smithi028.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:21 /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:51.466 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:51.817 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:21 /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:51.817 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:52.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:21 /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:52.204 DEBUG:teuthology.orchestra.run.smithi028:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:52.224 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:52.242 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:21:52.566 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T09:21:52.567 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:21:52.570 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:21:52.572 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:21:54.112 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:21:54.147 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:21:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:22:26.949 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:22:26.949 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:22:26.949 INFO:teuthology.orchestra.run.smithi028.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:22:26.949 INFO:teuthology.orchestra.run.smithi028.stdout: "repo_digests": [ 2024-09-19T09:22:26.950 INFO:teuthology.orchestra.run.smithi028.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:22:26.950 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:22:26.950 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:22:40.617 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-19T09:22:40.618 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T09:22:41.887 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T09:22:41.914 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph 2024-09-19T09:22:41.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-09-19T09:22:41.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-09-19T09:22:42.012 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /etc/ceph 2024-09-19T09:22:42.042 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-09-19T09:22:42.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-09-19T09:22:42.097 INFO:tasks.cephadm:Writing seed config... 2024-09-19T09:22:42.098 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T09:22:42.099 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T09:22:42.100 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T09:22:42.101 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T09:22:42.102 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T09:22:42.103 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:22:42.103 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T09:22:42.122 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9a2e9aa4-7668-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T09:22:42.122 DEBUG:teuthology.orchestra.run.smithi028:mon.a> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T09:22:42.165 DEBUG:teuthology.orchestra.run.smithi028:mgr.x> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T09:22:42.208 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T09:22:42.208 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.28 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:22:42.422 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:22:42.422 INFO:teuthology.orchestra.run.smithi028.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '9a2e9aa4-7668-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.28', '--skip-admin-label'] 2024-09-19T09:22:42.422 INFO:teuthology.orchestra.run.smithi028.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T09:22:42.423 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying podman|docker is present... 2024-09-19T09:22:42.446 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 5.2.2 2024-09-19T09:22:42.447 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying lvm2 is present... 2024-09-19T09:22:42.447 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying time synchronization is in place... 2024-09-19T09:22:42.454 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T09:22:42.455 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T09:22:42.461 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T09:22:42.461 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:22:42.468 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout enabled 2024-09-19T09:22:42.474 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout active 2024-09-19T09:22:42.474 INFO:teuthology.orchestra.run.smithi028.stdout:Unit chronyd.service is enabled and running 2024-09-19T09:22:42.475 INFO:teuthology.orchestra.run.smithi028.stdout:Repeating the final host check... 2024-09-19T09:22:42.498 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout 5.2.2 2024-09-19T09:22:42.498 INFO:teuthology.orchestra.run.smithi028.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T09:22:42.498 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl is present 2024-09-19T09:22:42.498 INFO:teuthology.orchestra.run.smithi028.stdout:lvcreate is present 2024-09-19T09:22:42.505 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T09:22:42.505 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T09:22:42.511 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T09:22:42.511 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:22:42.518 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout enabled 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout active 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:Unit chronyd.service is enabled and running 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:Host looks OK 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:Cluster fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:Acquiring lock 139872358822912 on /run/cephadm/9a2e9aa4-7668-11ef-bceb-c7b262605968.lock 2024-09-19T09:22:42.525 INFO:teuthology.orchestra.run.smithi028.stdout:Lock 139872358822912 acquired on /run/cephadm/9a2e9aa4-7668-11ef-bceb-c7b262605968.lock 2024-09-19T09:22:42.526 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying IP 172.21.15.28 port 3300 ... 2024-09-19T09:22:42.526 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying IP 172.21.15.28 port 6789 ... 2024-09-19T09:22:42.526 INFO:teuthology.orchestra.run.smithi028.stdout:Base mon IP(s) is [172.21.15.28:3300, 172.21.15.28:6789], mon addrv is [v2:172.21.15.28:3300,v1:172.21.15.28:6789] 2024-09-19T09:22:42.529 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.28 metric 100 2024-09-19T09:22:42.530 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.28 metric 100 2024-09-19T09:22:42.532 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T09:22:42.532 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T09:22:42.532 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1785sec hoplimit 64 pref medium 2024-09-19T09:22:42.534 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T09:22:42.535 INFO:teuthology.orchestra.run.smithi028.stdout:Mon IP `172.21.15.28` is in CIDR network `172.21.0.0/20` 2024-09-19T09:22:42.536 INFO:teuthology.orchestra.run.smithi028.stdout:Mon IP `172.21.15.28` is in CIDR network `172.21.0.0/20` 2024-09-19T09:22:42.536 INFO:teuthology.orchestra.run.smithi028.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T09:22:42.536 INFO:teuthology.orchestra.run.smithi028.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T09:22:42.537 INFO:teuthology.orchestra.run.smithi028.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T09:22:42.985 INFO:teuthology.orchestra.run.smithi028.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T09:22:43.831 INFO:teuthology.orchestra.run.smithi028.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:22:43.831 INFO:teuthology.orchestra.run.smithi028.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:22:43.831 INFO:teuthology.orchestra.run.smithi028.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T09:22:44.636 INFO:teuthology.orchestra.run.smithi028.stdout:stat: stdout 167 167 2024-09-19T09:22:44.636 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial keys... 2024-09-19T09:22:45.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQBk7etm467IOBAAbs8OsmTg8ubWq9y+6yeWhw== 2024-09-19T09:22:46.347 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQBl7etm/EUyLhAA049TPlBYO9KpB9R/Y0a6YA== 2024-09-19T09:22:47.254 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph-authtool: stdout AQBm7etmxg+lKBAAi6RBkf0P/Eo6rRSqAAQ5DA== 2024-09-19T09:22:47.255 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial monmap... 2024-09-19T09:22:48.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool for a [v2:172.21.15.28:3300,v1:172.21.15.28:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = quincy 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: set fsid to 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:22:48.193 INFO:teuthology.orchestra.run.smithi028.stdout:Creating mon... 2024-09-19T09:22:52.205 INFO:teuthology.orchestra.run.smithi028.stdout:create mon.a on 2024-09-19T09:22:52.838 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T09:22:53.187 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T09:22:53.732 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target. 2024-09-19T09:22:53.732 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target. 2024-09-19T09:22:54.572 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a 2024-09-19T09:22:54.572 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T09:22:54.823 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target.wants/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@.service. 2024-09-19T09:22:55.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:55 smithi028 podman[30587]: 2024-09-19 09:22:55.031648872 +0000 UTC m=+0.016032489 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:22:56.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:55 smithi028 podman[30587]: 2024-09-19 09:22:55.952775498 +0000 UTC m=+0.937159088 container create 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:22:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:57 smithi028 podman[30587]: 2024-09-19 09:22:57.665327248 +0000 UTC m=+2.649710852 container init 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T09:22:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:57 smithi028 podman[30587]: 2024-09-19 09:22:57.669888678 +0000 UTC m=+2.654272269 container start 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T09:22:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:57 smithi028 ceph-mon[30606]: mkfs 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:57 smithi028 ceph-mon[30606]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:22:58.102 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:22:58.102 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-09-19T09:22:58.110 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:22:58.111 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:22:58.111 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-09-19T09:22:58.111 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T09:22:58.111 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mon to start... 2024-09-19T09:22:58.111 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mon... 2024-09-19T09:22:58.337 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 bash[30587]: 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 2024-09-19T09:22:58.338 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 systemd[1]: Started Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: monmap epoch 1 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: last_changed 2024-09-19T09:22:47.619360+0000 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: min_mon_release 19 (squid) 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: election_strategy: 1 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: fsmap 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T09:22:58.972 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: mgrmap e1: no daemons active 2024-09-19T09:22:58.973 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:58 smithi028 ceph-mon[30606]: from='client.? 172.21.15.28:0/71971296' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout id: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout services: 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.887446s) 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T09:22:59.231 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout data: 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.538+0000 7fbde15fc640 1 Processor -- start 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.538+0000 7fbde15fc640 1 -- start start 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.538+0000 7fbde15fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:59.232 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.538+0000 7fbde15fc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc108200 con 0x7fbddc107830 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.539+0000 7fbddaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.539+0000 7fbddaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50426/0 (socket says 172.21.15.28:50426) 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.539+0000 7fbddaffd640 1 -- 172.21.15.28:0/126270898 learned_addr learned my addr 172.21.15.28:0/126270898 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.539+0000 7fbddaffd640 1 -- 172.21.15.28:0/126270898 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc1089e0 con 0x7fbddc107830 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.541+0000 7fbddaffd640 1 --2- 172.21.15.28:0/126270898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e80 tx=0x7fbdcc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1de23eb56c0feb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.541+0000 7fbdd9ffb640 1 -- 172.21.15.28:0/126270898 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdcc036070 con 0x7fbddc107830 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.541+0000 7fbdd9ffb640 1 -- 172.21.15.28:0/126270898 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc031400 con 0x7fbddc107830 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.541+0000 7fbdd9ffb640 1 -- 172.21.15.28:0/126270898 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdcc031700 con 0x7fbddc107830 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 -- 172.21.15.28:0/126270898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 msgr2=0x7fbddc107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 --2- 172.21.15.28:0/126270898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e80 tx=0x7fbdcc02a7a0 comp rx=0 tx=0).stop 2024-09-19T09:22:59.233 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 -- 172.21.15.28:0/126270898 shutdown_connections 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 --2- 172.21.15.28:0/126270898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc107c30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 -- 172.21.15.28:0/126270898 >> 172.21.15.28:0/126270898 conn(0x7fbddc103040 msgr2=0x7fbddc105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 -- 172.21.15.28:0/126270898 shutdown_connections 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.542+0000 7fbde15fc640 1 -- 172.21.15.28:0/126270898 wait complete. 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbde15fc640 1 Processor -- start 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbde15fc640 1 -- start start 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbde15fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbde15fc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc108f80 con 0x7fbddc107830 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbddaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbddaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50440/0 (socket says 172.21.15.28:50440) 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.543+0000 7fbddaffd640 1 -- 172.21.15.28:0/71971296 learned_addr learned my addr 172.21.15.28:0/71971296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:22:59.234 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbddaffd640 1 -- 172.21.15.28:0/71971296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc19b4e0 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbddaffd640 1 --2- 172.21.15.28:0/71971296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034040 tx=0x7fbdcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdcc041070 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc03c650 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdcc036040 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbddc19b770 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc19e460 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbdcc044070 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.544+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbdcc046c10 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.545+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbddc1095d0 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.547+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbdcc03c950 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.609+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fbddc19e9c0 con 0x7fbddc107830 2024-09-19T09:22:59.235 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.610+0000 7fbdd3fff640 1 -- 172.21.15.28:0/71971296 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fbdcc03cb30 con 0x7fbddc107830 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 msgr2=0x7fbddc19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 --2- 172.21.15.28:0/71971296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034040 tx=0x7fbdcc004a40 comp rx=0 tx=0).stop 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 shutdown_connections 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 --2- 172.21.15.28:0/71971296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbddc107830 0x7fbddc19afa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 >> 172.21.15.28:0/71971296 conn(0x7fbddc103040 msgr2=0x7fbddc18f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 shutdown_connections 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:58.611+0000 7fbde15fc640 1 -- 172.21.15.28:0/71971296 wait complete. 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:mon is available 2024-09-19T09:22:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T09:23:00.002 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:59 smithi028 ceph-mon[30606]: from='client.? 172.21.15.28:0/1073684227' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T09:23:00.002 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:22:59 smithi028 ceph-mon[30606]: from='client.? 172.21.15.28:0/1073684227' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout fsid = 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.28:3300,v1:172.21.15.28:6789] 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T09:23:00.722 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.653+0000 7f35b3a91640 1 Processor -- start 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b3a91640 1 -- start start 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b3a91640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b3a91640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac107080 con 0x7f35ac1066b0 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b182d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.723 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b182d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50456/0 (socket says 172.21.15.28:50456) 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.654+0000 7f35b182d640 1 -- 172.21.15.28:0/2810870942 learned_addr learned my addr 172.21.15.28:0/2810870942 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.655+0000 7f35b182d640 1 -- 172.21.15.28:0/2810870942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1078b0 con 0x7f35ac1066b0 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.655+0000 7f35b182d640 1 --2- 172.21.15.28:0/2810870942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f359c009e80 tx=0x7f359c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f31c3577b5aaca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b082b640 1 -- 172.21.15.28:0/2810870942 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f359c036070 con 0x7f35ac1066b0 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b082b640 1 -- 172.21.15.28:0/2810870942 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c031400 con 0x7f35ac1066b0 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b082b640 1 -- 172.21.15.28:0/2810870942 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f359c031700 con 0x7f35ac1066b0 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 -- 172.21.15.28:0/2810870942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 msgr2=0x7f35ac106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 --2- 172.21.15.28:0/2810870942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f359c009e80 tx=0x7f359c02a7a0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 -- 172.21.15.28:0/2810870942 shutdown_connections 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 --2- 172.21.15.28:0/2810870942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac106ab0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 -- 172.21.15.28:0/2810870942 >> 172.21.15.28:0/2810870942 conn(0x7f35ac101e60 msgr2=0x7f35ac104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.724 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 -- 172.21.15.28:0/2810870942 shutdown_connections 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.656+0000 7f35b3a91640 1 -- 172.21.15.28:0/2810870942 wait complete. 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b3a91640 1 Processor -- start 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b3a91640 1 -- start start 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b3a91640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b3a91640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac107e50 con 0x7f35ac1066b0 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b182d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b182d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50472/0 (socket says 172.21.15.28:50472) 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.657+0000 7f35b182d640 1 -- 172.21.15.28:0/1073684227 learned_addr learned my addr 172.21.15.28:0/1073684227 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35b182d640 1 -- 172.21.15.28:0/1073684227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19d730 con 0x7f35ac1066b0 2024-09-19T09:23:00.725 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35b182d640 1 --2- 172.21.15.28:0/1073684227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f359c034040 tx=0x7f359c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f359c041070 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c03c650 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19d9c0 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f359c036040 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1a06b0 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.658+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f359c044070 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.659+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f359c046c10 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.659+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac1084a0 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.661+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f359c03c950 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.722+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f35ac1a1dd0 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.725+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f359c03cb30 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35aaffd640 1 -- 172.21.15.28:0/1073684227 <== mon.0 v2:172.21.15.28:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f359c031400 con 0x7f35ac1066b0 2024-09-19T09:23:00.726 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 msgr2=0x7f35ac19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 --2- 172.21.15.28:0/1073684227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f359c034040 tx=0x7f359c004a40 comp rx=0 tx=0).stop 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 shutdown_connections 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 --2- 172.21.15.28:0/1073684227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35ac1066b0 0x7f35ac19d1f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 >> 172.21.15.28:0/1073684227 conn(0x7f35ac101e60 msgr2=0x7f35ac07b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 shutdown_connections 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:22:59.726+0000 7f35b3a91640 1 -- 172.21.15.28:0/1073684227 wait complete. 2024-09-19T09:23:00.727 INFO:teuthology.orchestra.run.smithi028.stdout:Generating new minimal ceph.conf... 2024-09-19T09:23:01.477 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:01 smithi028 ceph-mon[30606]: from='client.? 172.21.15.28:0/4195742430' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:01.738 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f6589c9b640 1 Processor -- start 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f6589c9b640 1 -- start start 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f6589c9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f6589c9b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f658407be20 con 0x7f65841060c0 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f65837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f65837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50480/0 (socket says 172.21.15.28:50480) 2024-09-19T09:23:01.742 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f65837fe640 1 -- 172.21.15.28:0/1983976943 learned_addr learned my addr 172.21.15.28:0/1983976943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.153+0000 7f65837fe640 1 -- 172.21.15.28:0/1983976943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584106a00 con 0x7f65841060c0 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.154+0000 7f65837fe640 1 --2- 172.21.15.28:0/1983976943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6574009820 tx=0x7f657402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9744a04957de5874 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.154+0000 7f65827fc640 1 -- 172.21.15.28:0/1983976943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6574037070 con 0x7f65841060c0 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f65827fc640 1 -- 172.21.15.28:0/1983976943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6574032900 con 0x7f65841060c0 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f65827fc640 1 -- 172.21.15.28:0/1983976943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6574030850 con 0x7f65841060c0 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 -- 172.21.15.28:0/1983976943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 msgr2=0x7f65841064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 --2- 172.21.15.28:0/1983976943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6574009820 tx=0x7f657402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 -- 172.21.15.28:0/1983976943 shutdown_connections 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 --2- 172.21.15.28:0/1983976943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f65841064c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 -- 172.21.15.28:0/1983976943 >> 172.21.15.28:0/1983976943 conn(0x7f6584101f10 msgr2=0x7f6584104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 -- 172.21.15.28:0/1983976943 shutdown_connections 2024-09-19T09:23:01.743 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.155+0000 7f6589c9b640 1 -- 172.21.15.28:0/1983976943 wait complete. 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f6589c9b640 1 Processor -- start 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f6589c9b640 1 -- start start 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f6589c9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f6589c9b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6584107b00 con 0x7f65841060c0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f65837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f65837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50482/0 (socket says 172.21.15.28:50482) 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.156+0000 7f65837fe640 1 -- 172.21.15.28:0/4195742430 learned_addr learned my addr 172.21.15.28:0/4195742430 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f65837fe640 1 -- 172.21.15.28:0/4195742430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658419df20 con 0x7f65841060c0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f65837fe640 1 --2- 172.21.15.28:0/4195742430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65740097f0 tx=0x7f6574030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6574037040 con 0x7f65841060c0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6574040070 con 0x7f65841060c0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658419e1b0 con 0x7f65841060c0 2024-09-19T09:23:01.744 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f657403a390 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65841a0ea0 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f657403a630 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.157+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65740459f0 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.158+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65841064c0 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.161+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6574045460 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.221+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6584196e50 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.221+0000 7f6580ff9640 1 -- 172.21.15.28:0/4195742430 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f657403b430 con 0x7f65841060c0 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.222+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 msgr2=0x7f658419d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.222+0000 7f6589c9b640 1 --2- 172.21.15.28:0/4195742430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65740097f0 tx=0x7f6574030d80 comp rx=0 tx=0).stop 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.222+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 shutdown_connections 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.222+0000 7f6589c9b640 1 --2- 172.21.15.28:0/4195742430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65841060c0 0x7f658419d9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:01.745 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.222+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 >> 172.21.15.28:0/4195742430 conn(0x7f6584101f10 msgr2=0x7f6584191cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:01.746 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.223+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 shutdown_connections 2024-09-19T09:23:01.746 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:01.223+0000 7f6589c9b640 1 -- 172.21.15.28:0/4195742430 wait complete. 2024-09-19T09:23:01.746 INFO:teuthology.orchestra.run.smithi028.stdout:Restarting the monitor... 2024-09-19T09:23:02.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:01 smithi028 systemd[1]: Stopping Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:23:02.439 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30602]: 2024-09-19T09:23:02.012+0000 7f7c07161640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:23:02.439 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30602]: 2024-09-19T09:23:02.012+0000 7f7c07161640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T09:23:02.439 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 podman[30847]: 2024-09-19 09:23:02.136861744 +0000 UTC m=+0.222414921 container died 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:23:02.756 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 podman[30847]: 2024-09-19 09:23:02.538295627 +0000 UTC m=+0.623848803 container cleanup 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:23:02.756 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 bash[30847]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a 2024-09-19T09:23:03.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 podman[30861]: 2024-09-19 09:23:02.755862533 +0000 UTC m=+0.607899459 container remove 1ac0e569b44b66755e4078ce6d8ae530d6d29d34b03350f7e3672730b356f7a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:23:03.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T09:23:03.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 systemd[1]: Stopped Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:23:03.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:02 smithi028 systemd[1]: Starting Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:23:03.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:03 smithi028 podman[30953]: 2024-09-19 09:23:03.094595257 +0000 UTC m=+0.016043618 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:23:03.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:03 smithi028 podman[30953]: 2024-09-19 09:23:03.357719865 +0000 UTC m=+0.279168235 container create 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:23:05.096 INFO:teuthology.orchestra.run.smithi028.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T09:23:05.306 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 podman[30953]: 2024-09-19 09:23:05.016414538 +0000 UTC m=+1.937862909 container init 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:23:05.306 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 podman[30953]: 2024-09-19 09:23:05.020666194 +0000 UTC m=+1.942114556 container start 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: pidfile_write: ignore empty --pid-file 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: load: jerasure load: lrc 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: RocksDB version: 7.9.2 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Git sha 0 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: DB SUMMARY 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: DB Session ID: MLO4QBUDO61EFAN12XRW 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: CURRENT file: CURRENT 2024-09-19T09:23:05.307 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106770 ; 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.error_if_exists: 0 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.create_if_missing: 0 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.paranoid_checks: 1 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.env: 0x55b1808b7940 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T09:23:05.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.info_log: 0x55b18152b6e0 2024-09-19T09:23:05.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T09:23:05.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.statistics: (nil) 2024-09-19T09:23:05.310 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.use_fsync: 0 2024-09-19T09:23:05.310 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_log_file_size: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_fallocate: 1 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.use_direct_reads: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.db_log_dir: 2024-09-19T09:23:05.311 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.wal_dir: 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.write_buffer_manager: 0x55b18153adc0 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T09:23:05.312 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.unordered_write: 0 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.row_cache: None 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.wal_filter: None 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T09:23:05.313 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.two_write_queues: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.wal_compression: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.atomic_flush: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.log_readahead_size: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T09:23:05.314 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_background_jobs: 2 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_background_compactions: -1 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_subcompactions: 1 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T09:23:05.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_open_files: -1 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_background_flushes: -1 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Compression algorithms supported: 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kZSTD supported: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kXpressCompression supported: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kBZip2Compression supported: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kLZ4Compression supported: 1 2024-09-19T09:23:05.316 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kZlibCompression supported: 1 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: kSnappyCompression supported: 1 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.merge_operator: 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_filter: None 2024-09-19T09:23:05.317 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b18152b320) 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: cache_index_and_filter_blocks: 1 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: pin_top_level_index_and_filter: 1 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: index_type: 0 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: data_block_index_type: 0 2024-09-19T09:23:05.318 INFO:journalctl@ceph.mon.a.smithi028.stdout: index_shortening: 1 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: checksum: 4 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: no_block_cache: 0 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_cache: 0x55b18151f1f0 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_cache_name: BinnedLRUCache 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_cache_options: 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: capacity : 536870912 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: num_shard_bits : 4 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: strict_capacity_limit : 0 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: high_pri_pool_ratio: 0.000 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_cache_compressed: (nil) 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: persistent_cache: (nil) 2024-09-19T09:23:05.319 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_size: 4096 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_size_deviation: 10 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_restart_interval: 16 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: index_block_restart_interval: 1 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: metadata_block_size: 4096 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: partition_filters: 0 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: use_delta_encoding: 1 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: filter_policy: bloomfilter 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: whole_key_filtering: 1 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: verify_compression: 0 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: read_amp_bytes_per_bit: 0 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: format_version: 5 2024-09-19T09:23:05.320 INFO:journalctl@ceph.mon.a.smithi028.stdout: enable_index_compression: 1 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout: block_align: 0 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout: max_auto_readahead_size: 262144 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout: prepopulate_block_cache: 0 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout: initial_auto_readahead_size: 8192 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression: NoCompression 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.num_levels: 7 2024-09-19T09:23:05.321 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T09:23:05.322 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T09:23:05.323 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T09:23:05.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T09:23:05.325 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.inplace_update_support: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.bloom_locality: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.max_successive_merges: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T09:23:05.326 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.ttl: 2592000 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enable_blob_files: false 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.min_blob_size: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T09:23:05.327 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fae1c64e-ee7d-4c3d-b598-d67b298689fc 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737785052740, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737785054030, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101704, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99589, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14671, "raw_average_key_size": 50, "raw_value_size": 91644, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726737785, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fae1c64e-ee7d-4c3d-b598-d67b298689fc", "db_session_id": "MLO4QBUDO61EFAN12XRW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737785054123, "job": 1, "event": "recovery_finished"} 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b18154ce00 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: DB pointer 0x55b181648000 2024-09-19T09:23:05.328 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: ** DB Stats ** 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:23:05.329 INFO:journalctl@ceph.mon.a.smithi028.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: ** Compaction Stats [default] ** 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: L0 2/0 101.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: Sum 2/0 101.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:23:05.330 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: ** Compaction Stats [default] ** 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 116.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.331 INFO:journalctl@ceph.mon.a.smithi028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Cumulative compaction: 0.00 GB write, 24.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Interval compaction: 0.00 GB write, 24.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Block cache BinnedLRUCache@0x55b18151f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-19T09:23:05.332 INFO:journalctl@ceph.mon.a.smithi028.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: starting mon.a rank 0 at public addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] at bind addrs [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???) e1 preinit fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).mds e1 new map 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).mds e1 print_map 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: e1 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: btime 2024-09-19T09:22:57:723574+0000 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: legacy client fscid: -1 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: 2024-09-19T09:23:05.333 INFO:journalctl@ceph.mon.a.smithi028.stdout: No filesystems configured 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 bash[30953]: 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 systemd[1]: Started Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: monmap epoch 1 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: last_changed 2024-09-19T09:22:47.619360+0000 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T09:23:05.334 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: election_strategy: 1 2024-09-19T09:23:05.335 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:23:05.335 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: fsmap 2024-09-19T09:23:05.335 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T09:23:05.335 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:05 smithi028 ceph-mon[30972]: mgrmap e1: no daemons active 2024-09-19T09:23:06.183 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.514+0000 7f967455d640 1 Processor -- start 2024-09-19T09:23:06.187 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.514+0000 7f967455d640 1 -- start start 2024-09-19T09:23:06.187 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f967455d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f967455d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f966c07be00 con 0x7f966c106070 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f96722f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f96722f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33174/0 (socket says 172.21.15.28:33174) 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f96722f9640 1 -- 172.21.15.28:0/4078856504 learned_addr learned my addr 172.21.15.28:0/4078856504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.515+0000 7f96722f9640 1 -- 172.21.15.28:0/4078856504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c1069b0 con 0x7f966c106070 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.517+0000 7f96722f9640 1 --2- 172.21.15.28:0/4078856504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9668009e80 tx=0x7f966802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be1b6b96fc21926 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.517+0000 7f96712f7640 1 -- 172.21.15.28:0/4078856504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9668036070 con 0x7f966c106070 2024-09-19T09:23:06.188 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.517+0000 7f96712f7640 1 -- 172.21.15.28:0/4078856504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96680318c0 con 0x7f966c106070 2024-09-19T09:23:06.189 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.517+0000 7f96712f7640 1 -- 172.21.15.28:0/4078856504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9668007ab0 con 0x7f966c106070 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 -- 172.21.15.28:0/4078856504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 msgr2=0x7f966c106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 --2- 172.21.15.28:0/4078856504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9668009e80 tx=0x7f966802a7a0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 -- 172.21.15.28:0/4078856504 shutdown_connections 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 --2- 172.21.15.28:0/4078856504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c106470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 -- 172.21.15.28:0/4078856504 >> 172.21.15.28:0/4078856504 conn(0x7f966c101e60 msgr2=0x7f966c104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 -- 172.21.15.28:0/4078856504 shutdown_connections 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.518+0000 7f967455d640 1 -- 172.21.15.28:0/4078856504 wait complete. 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f967455d640 1 Processor -- start 2024-09-19T09:23:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f967455d640 1 -- start start 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f967455d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f967455d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f966c107ab0 con 0x7f966c106070 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f96722f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f96722f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33182/0 (socket says 172.21.15.28:33182) 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f96722f9640 1 -- 172.21.15.28:0/4015376849 learned_addr learned my addr 172.21.15.28:0/4015376849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f96722f9640 1 -- 172.21.15.28:0/4015376849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c19def0 con 0x7f966c106070 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.519+0000 7f96722f9640 1 --2- 172.21.15.28:0/4015376849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9668031ca0 tx=0x7f9668031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9668036050 con 0x7f966c106070 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9668042070 con 0x7f966c106070 2024-09-19T09:23:06.191 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c19e180 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966803d820 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1a0e70 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f966803dad0 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.520+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9668007e00 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.521+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966c106470 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.523+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f96680072f0 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.583+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f966c02d9c0 con 0x7f966c106070 2024-09-19T09:23:06.192 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.584+0000 7f96637fe640 1 -- 172.21.15.28:0/4015376849 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9668047460 con 0x7f966c106070 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 msgr2=0x7f966c19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 --2- 172.21.15.28:0/4015376849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9668031ca0 tx=0x7f9668031cd0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 shutdown_connections 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 --2- 172.21.15.28:0/4015376849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c106070 0x7f966c19d9b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 >> 172.21.15.28:0/4015376849 conn(0x7f966c101e60 msgr2=0x7f966c194670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 shutdown_connections 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:05.585+0000 7f967455d640 1 -- 172.21.15.28:0/4015376849 wait complete. 2024-09-19T09:23:06.193 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T09:23:06.214 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:23:06.214 INFO:teuthology.orchestra.run.smithi028.stdout:Creating mgr... 2024-09-19T09:23:06.214 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T09:23:06.215 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T09:23:06.492 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x 2024-09-19T09:23:06.492 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T09:23:06.650 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968.target.wants/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@.service. 2024-09-19T09:23:06.884 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:06 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4015376849' entity='client.admin' 2024-09-19T09:23:06.884 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:06 smithi028 systemd[1]: Starting Ceph mgr.x for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:23:07.160 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:06 smithi028 podman[31184]: 2024-09-19 09:23:06.786024116 +0000 UTC m=+0.016279931 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:23:07.160 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:06 smithi028 podman[31184]: 2024-09-19 09:23:06.899617298 +0000 UTC m=+0.129873100 container create 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T09:23:07.161 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 podman[31184]: 2024-09-19 09:23:07.101311577 +0000 UTC m=+0.331567397 container init 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T09:23:07.161 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 podman[31184]: 2024-09-19 09:23:07.104079643 +0000 UTC m=+0.334335454 container start 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:23:07.161 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 bash[31184]: 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 2024-09-19T09:23:07.177 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:23:07.177 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-09-19T09:23:07.186 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:23:07.186 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:23:07.186 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-09-19T09:23:07.186 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T09:23:07.197 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:23:07.197 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr to start... 2024-09-19T09:23:07.206 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr... 2024-09-19T09:23:07.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 systemd[1]: Started Ceph mgr.x for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:23:07.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.240+0000 7ffbea6d7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:23:07.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.299+0000 7ffbea6d7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:07.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.460+0000 7ffbea6d7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:23:07.740 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.533+0000 7ffbea6d7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:23:07.740 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.692+0000 7ffbea6d7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:08.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3296875966' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:23:08.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:07.843+0000 7ffbea6d7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:23:08.303 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:23:08.316 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:23:08.317 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:23:08.318 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:22:57:723574+0000", 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:23:08.319 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.320 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:22:57.724340+0000", 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.667+0000 7fe313078640 1 Processor -- start 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe313078640 1 -- start start 2024-09-19T09:23:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe313078640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe313078640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c106460 con 0x7fe30c105a90 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe310e14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe310e14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33212/0 (socket says 172.21.15.28:33212) 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe310e14640 1 -- 172.21.15.28:0/1805644709 learned_addr learned my addr 172.21.15.28:0/1805644709 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.668+0000 7fe310e14640 1 -- 172.21.15.28:0/1805644709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c1065e0 con 0x7fe30c105a90 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.669+0000 7fe310e14640 1 --2- 172.21.15.28:0/1805644709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009820 tx=0x7fe2fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc075bee9e828a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.669+0000 7fe30b7fe640 1 -- 172.21.15.28:0/1805644709 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc037070 con 0x7fe30c105a90 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.669+0000 7fe30b7fe640 1 -- 172.21.15.28:0/1805644709 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2fc032900 con 0x7fe30c105a90 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.669+0000 7fe30b7fe640 1 -- 172.21.15.28:0/1805644709 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc030850 con 0x7fe30c105a90 2024-09-19T09:23:08.322 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 -- 172.21.15.28:0/1805644709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 msgr2=0x7fe30c105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 --2- 172.21.15.28:0/1805644709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009820 tx=0x7fe2fc02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 -- 172.21.15.28:0/1805644709 shutdown_connections 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 --2- 172.21.15.28:0/1805644709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c105e90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 -- 172.21.15.28:0/1805644709 >> 172.21.15.28:0/1805644709 conn(0x7fe30c101620 msgr2=0x7fe30c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 -- 172.21.15.28:0/1805644709 shutdown_connections 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.670+0000 7fe313078640 1 -- 172.21.15.28:0/1805644709 wait complete. 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe313078640 1 Processor -- start 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe313078640 1 -- start start 2024-09-19T09:23:08.323 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe313078640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe313078640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c109970 con 0x7fe30c105a90 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe310e14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe310e14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33218/0 (socket says 172.21.15.28:33218) 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe310e14640 1 -- 172.21.15.28:0/3296875966 learned_addr learned my addr 172.21.15.28:0/3296875966 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.671+0000 7fe310e14640 1 -- 172.21.15.28:0/3296875966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c19bd10 con 0x7fe30c105a90 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe310e14640 1 --2- 172.21.15.28:0/3296875966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0097f0 tx=0x7fe2fc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc042070 con 0x7fe30c105a90 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2fc032ca0 con 0x7fe30c105a90 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c19c000 con 0x7fe30c105a90 2024-09-19T09:23:08.324 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc037040 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c19ec90 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2fc045070 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.672+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2fc0479e0 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.673+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c105f10 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.676+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe2fc03da60 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.738+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe30c19f270 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.738+0000 7fe309ffb640 1 -- 172.21.15.28:0/3296875966 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe2fc03d430 con 0x7fe30c105a90 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.739+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 msgr2=0x7fe30c19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.739+0000 7fe313078640 1 --2- 172.21.15.28:0/3296875966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0097f0 tx=0x7fe2fc030ec0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.325 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.739+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 shutdown_connections 2024-09-19T09:23:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.739+0000 7fe313078640 1 --2- 172.21.15.28:0/3296875966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe30c105a90 0x7fe30c19b7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.739+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 >> 172.21.15.28:0/3296875966 conn(0x7fe30c101620 msgr2=0x7fe30c102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.740+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 shutdown_connections 2024-09-19T09:23:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:07.740+0000 7fe313078640 1 -- 172.21.15.28:0/3296875966 wait complete. 2024-09-19T09:23:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:mgr not available, waiting (1/15)... 2024-09-19T09:23:08.440 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:08.178+0000 7ffbea6d7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:23:08.440 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:08.242+0000 7ffbea6d7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:23:08.440 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:08.438+0000 7ffbea6d7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:23:09.763 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:09.362+0000 7ffbea6d7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:23:09.763 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:09.432+0000 7ffbea6d7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:23:10.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.185+0000 7ffbea6d7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:23:10.462 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.305+0000 7ffbea6d7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:23:10.756 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.460+0000 7ffbea6d7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:10.756 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.524+0000 7ffbea6d7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:23:10.756 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.682+0000 7ffbea6d7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:23:10.756 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.754+0000 7ffbea6d7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:23:11.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.821+0000 7ffbea6d7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:23:11.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:10.992+0000 7ffbea6d7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:23:11.286 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2091851699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:23:11.286 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:11.063+0000 7ffbea6d7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:23:11.286 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:11.133+0000 7ffbea6d7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:23:11.286 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:11.219+0000 7ffbea6d7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:11.286 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:11.284+0000 7ffbea6d7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:23:11.614 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:11.346+0000 7ffbea6d7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:23:11.618 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:11.619 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:23:11.620 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:23:11.628 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:22:57:723574+0000", 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:23:11.629 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:22:57.724340+0000", 2024-09-19T09:23:11.630 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc39de29640 1 Processor -- start 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc39de29640 1 -- start start 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc39de29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc39de29640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc39807be60 con 0x7fc398108220 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33234/0 (socket says 172.21.15.28:33234) 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.946+0000 7fc3977fe640 1 -- 172.21.15.28:0/584260866 learned_addr learned my addr 172.21.15.28:0/584260866 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.947+0000 7fc3977fe640 1 -- 172.21.15.28:0/584260866 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39810ab50 con 0x7fc398108220 2024-09-19T09:23:11.631 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.947+0000 7fc3977fe640 1 --2- 172.21.15.28:0/584260866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc388009820 tx=0x7fc38802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8064e2ebbd4a69d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.948+0000 7fc3967fc640 1 -- 172.21.15.28:0/584260866 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388037070 con 0x7fc398108220 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.948+0000 7fc3967fc640 1 -- 172.21.15.28:0/584260866 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc388032900 con 0x7fc398108220 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.948+0000 7fc3967fc640 1 -- 172.21.15.28:0/584260866 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388030850 con 0x7fc398108220 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 -- 172.21.15.28:0/584260866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 msgr2=0x7fc39810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 --2- 172.21.15.28:0/584260866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc388009820 tx=0x7fc38802a270 comp rx=0 tx=0).stop 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 -- 172.21.15.28:0/584260866 shutdown_connections 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 --2- 172.21.15.28:0/584260866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39810a610 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 -- 172.21.15.28:0/584260866 >> 172.21.15.28:0/584260866 conn(0x7fc398101e60 msgr2=0x7fc398104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 -- 172.21.15.28:0/584260866 shutdown_connections 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 -- 172.21.15.28:0/584260866 wait complete. 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.949+0000 7fc39de29640 1 Processor -- start 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc39de29640 1 -- start start 2024-09-19T09:23:11.632 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc39de29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc39de29640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810bbf0 con 0x7fc398108220 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33240/0 (socket says 172.21.15.28:33240) 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc3977fe640 1 -- 172.21.15.28:0/2091851699 learned_addr learned my addr 172.21.15.28:0/2091851699 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.950+0000 7fc3977fe640 1 -- 172.21.15.28:0/2091851699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39819bd10 con 0x7fc398108220 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc3977fe640 1 --2- 172.21.15.28:0/2091851699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc388032ce0 tx=0x7fc388032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388042070 con 0x7fc398108220 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39819bfa0 con 0x7fc398108220 2024-09-19T09:23:11.633 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3880308c0 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388037040 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39819ec90 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc388045070 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.951+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc388047bc0 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.952+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc398108680 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:10.954+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc38803c7e0 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.018+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc39819f210 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.018+0000 7fc394ff9640 1 -- 172.21.15.28:0/2091851699 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc38803c9c0 con 0x7fc398108220 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.019+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 msgr2=0x7fc39819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.019+0000 7fc39de29640 1 --2- 172.21.15.28:0/2091851699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc388032ce0 tx=0x7fc388032d10 comp rx=0 tx=0).stop 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.019+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 shutdown_connections 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.019+0000 7fc39de29640 1 --2- 172.21.15.28:0/2091851699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc398108220 0x7fc39819b7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.019+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 >> 172.21.15.28:0/2091851699 conn(0x7fc398101e60 msgr2=0x7fc398102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:11.635 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.020+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 shutdown_connections 2024-09-19T09:23:11.635 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:11.020+0000 7fc39de29640 1 -- 172.21.15.28:0/2091851699 wait complete. 2024-09-19T09:23:11.635 INFO:teuthology.orchestra.run.smithi028.stdout:mgr not available, waiting (2/15)... 2024-09-19T09:23:12.513 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:12.178+0000 7ffbea6d7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: Activating manager daemon x 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: mgrmap e2: x(active, starting, since 0.0035137s) 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:23:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:23:12.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' 2024-09-19T09:23:12.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' 2024-09-19T09:23:12.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:12 smithi028 ceph-mon[30972]: from='mgr.14100 172.21.15.28:0/3431291722' entity='mgr.x' 2024-09-19T09:23:14.297 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:14 smithi028 ceph-mon[30972]: mgrmap e3: x(active, since 1.00737s) 2024-09-19T09:23:14.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:14.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:23:14.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:23:14.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:23:14.893 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:23:14.893 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:23:14.893 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:23:14.894 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:22:57:723574+0000", 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:23:14.895 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:22:57.724340+0000", 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.038+0000 7f2aef631640 1 Processor -- start 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.038+0000 7f2aef631640 1 -- start start 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aef631640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aef631640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae8107080 con 0x7f2ae81066b0 2024-09-19T09:23:14.896 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aed3cd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aed3cd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46540/0 (socket says 172.21.15.28:46540) 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aed3cd640 1 -- 172.21.15.28:0/2890199070 learned_addr learned my addr 172.21.15.28:0/2890199070 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.039+0000 7f2aed3cd640 1 -- 172.21.15.28:0/2890199070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae81078b0 con 0x7f2ae81066b0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.040+0000 7f2aed3cd640 1 --2- 172.21.15.28:0/2890199070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009e80 tx=0x7f2ad802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c4e01817976efa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.040+0000 7f2ae7fff640 1 -- 172.21.15.28:0/2890199070 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ad8036070 con 0x7f2ae81066b0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.040+0000 7f2ae7fff640 1 -- 172.21.15.28:0/2890199070 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ad80318c0 con 0x7f2ae81066b0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.040+0000 7f2ae7fff640 1 -- 172.21.15.28:0/2890199070 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ad8007ab0 con 0x7f2ae81066b0 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 -- 172.21.15.28:0/2890199070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 msgr2=0x7f2ae8106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 --2- 172.21.15.28:0/2890199070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009e80 tx=0x7f2ad802a7a0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 -- 172.21.15.28:0/2890199070 shutdown_connections 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 --2- 172.21.15.28:0/2890199070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae8106ab0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 -- 172.21.15.28:0/2890199070 >> 172.21.15.28:0/2890199070 conn(0x7f2ae8101e60 msgr2=0x7f2ae8104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 -- 172.21.15.28:0/2890199070 shutdown_connections 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.041+0000 7f2aef631640 1 -- 172.21.15.28:0/2890199070 wait complete. 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aef631640 1 Processor -- start 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aef631640 1 -- start start 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aef631640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aef631640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae810ac60 con 0x7f2ae81066b0 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aed3cd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aed3cd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46544/0 (socket says 172.21.15.28:46544) 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aed3cd640 1 -- 172.21.15.28:0/3066330754 learned_addr learned my addr 172.21.15.28:0/3066330754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aed3cd640 1 -- 172.21.15.28:0/3066330754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae819df80 con 0x7f2ae81066b0 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.042+0000 7f2aed3cd640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ad802ad90 tx=0x7f2ad8031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.043+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ad8036050 con 0x7f2ae81066b0 2024-09-19T09:23:14.898 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.043+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ad8042070 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.043+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae819e210 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.043+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ad803d7e0 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.043+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae81a0f00 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f2ad803b610 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae810ade0 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2ae67fc640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 0x7f2ad003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ad8007e00 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2aecbcc640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 0x7f2ad003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.044+0000 7f2aecbcc640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 0x7f2ad003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098c0 tx=0x7f2adc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.049+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ad8042220 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.186+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2ad8047460 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.295+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2ae80008d0 con 0x7f2ae81066b0 2024-09-19T09:23:14.899 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.296+0000 7f2ae67fc640 1 -- 172.21.15.28:0/3066330754 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f2ad804d250 con 0x7f2ae81066b0 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 msgr2=0x7f2ad003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 0x7f2ad003b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098c0 tx=0x7f2adc006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 msgr2=0x7f2ae819da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ad802ad90 tx=0x7f2ad8031ca0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 shutdown_connections 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f2ad0039050 0x7f2ad003b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 --2- 172.21.15.28:0/3066330754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ae81066b0 0x7f2ae819da40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 >> 172.21.15.28:0/3066330754 conn(0x7f2ae8101e60 msgr2=0x7f2ae8102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 shutdown_connections 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:14.298+0000 7f2aef631640 1 -- 172.21.15.28:0/3066330754 wait complete. 2024-09-19T09:23:14.900 INFO:teuthology.orchestra.run.smithi028.stdout:mgr is available 2024-09-19T09:23:15.455 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:15 smithi028 ceph-mon[30972]: mgrmap e4: x(active, since 2s) 2024-09-19T09:23:15.455 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:15 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3066330754' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:23:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T09:23:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout fsid = 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.28:3300,v1:172.21.15.28:6789] 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T09:23:16.036 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.267+0000 7f1f46d57640 1 Processor -- start 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.267+0000 7f1f46d57640 1 -- start start 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.267+0000 7f1f46d57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.267+0000 7f1f46d57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401070f0 con 0x7f1f40106720 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.267+0000 7f1f44af3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.268+0000 7f1f44af3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46548/0 (socket says 172.21.15.28:46548) 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.268+0000 7f1f44af3640 1 -- 172.21.15.28:0/3971017413 learned_addr learned my addr 172.21.15.28:0/3971017413 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.268+0000 7f1f44af3640 1 -- 172.21.15.28:0/3971017413 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f40107920 con 0x7f1f40106720 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f44af3640 1 --2- 172.21.15.28:0/3971017413 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009820 tx=0x7f1f3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f2f51969a93de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.037 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f3f7fe640 1 -- 172.21.15.28:0/3971017413 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f34037070 con 0x7f1f40106720 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f3f7fe640 1 -- 172.21.15.28:0/3971017413 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f34032900 con 0x7f1f40106720 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f3f7fe640 1 -- 172.21.15.28:0/3971017413 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f34030850 con 0x7f1f40106720 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 -- 172.21.15.28:0/3971017413 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 msgr2=0x7f1f40106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 --2- 172.21.15.28:0/3971017413 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009820 tx=0x7f1f3402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 -- 172.21.15.28:0/3971017413 shutdown_connections 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 --2- 172.21.15.28:0/3971017413 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f40106b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 -- 172.21.15.28:0/3971017413 >> 172.21.15.28:0/3971017413 conn(0x7f1f40101f10 msgr2=0x7f1f40104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.269+0000 7f1f46d57640 1 -- 172.21.15.28:0/3971017413 shutdown_connections 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.270+0000 7f1f46d57640 1 -- 172.21.15.28:0/3971017413 wait complete. 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.270+0000 7f1f46d57640 1 Processor -- start 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.270+0000 7f1f46d57640 1 -- start start 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.270+0000 7f1f46d57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.038 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.270+0000 7f1f46d57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010acd0 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f44af3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f44af3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46550/0 (socket says 172.21.15.28:46550) 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f44af3640 1 -- 172.21.15.28:0/1031423048 learned_addr learned my addr 172.21.15.28:0/1031423048 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f44af3640 1 -- 172.21.15.28:0/1031423048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4019ac60 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f44af3640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009940 tx=0x7f1f34032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f34042070 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f340308c0 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4019aef0 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f34037040 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.271+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4019dbe0 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.272+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f1f34045070 con 0x7f1f40106720 2024-09-19T09:23:16.039 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.272+0000 7f1f3dffb640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 0x7f1f3003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.272+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4010ae50 con 0x7f1f40106720 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.273+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f3406c410 con 0x7f1f40106720 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.273+0000 7f1f3ffff640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 0x7f1f3003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.273+0000 7f1f3ffff640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 0x7f1f3003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0098c0 tx=0x7f1f2c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.278+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f34042d90 con 0x7f1f40106720 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.453+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1f40106ba0 con 0x7f1f40106720 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.455+0000 7f1f3dffb640 1 -- 172.21.15.28:0/1031423048 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f1f3406be30 con 0x7f1f40106720 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 msgr2=0x7f1f3003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 0x7f1f3003b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0098c0 tx=0x7f1f2c006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 msgr2=0x7f1f4019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009940 tx=0x7f1f34032ce0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.040 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 shutdown_connections 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f1f30039160 0x7f1f3003b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 --2- 172.21.15.28:0/1031423048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f40106720 0x7f1f4019a720 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 >> 172.21.15.28:0/1031423048 conn(0x7f1f40101f10 msgr2=0x7f1f40102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 shutdown_connections 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:15.457+0000 7f1f46d57640 1 -- 172.21.15.28:0/1031423048 wait complete. 2024-09-19T09:23:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling cephadm module... 2024-09-19T09:23:16.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:16 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1031423048' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T09:23:17.448 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3973959745' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T09:23:17.449 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ignoring --setuser ceph since I am not root 2024-09-19T09:23:17.449 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ignoring --setgroup ceph since I am not root 2024-09-19T09:23:17.449 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.337+0000 7fd52a8ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:23:17.449 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.398+0000 7fd52a8ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:17.758 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.553+0000 7fd52a8ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:23:17.758 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.623+0000 7fd52a8ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:23:17.764 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.464+0000 7f68f8ef8640 1 Processor -- start 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f8ef8640 1 -- start start 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f8ef8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f8ef8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410a3e0 con 0x7f68f4109a10 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46564/0 (socket says 172.21.15.28:46564) 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f259c640 1 -- 172.21.15.28:0/1273269414 learned_addr learned my addr 172.21.15.28:0/1273269414 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.465+0000 7f68f259c640 1 -- 172.21.15.28:0/1273269414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f410abc0 con 0x7f68f4109a10 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.466+0000 7f68f259c640 1 --2- 172.21.15.28:0/1273269414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009820 tx=0x7f68e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd64318f04565663 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.466+0000 7f68f1d9b640 1 -- 172.21.15.28:0/1273269414 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e4037070 con 0x7f68f4109a10 2024-09-19T09:23:17.765 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.466+0000 7f68f1d9b640 1 -- 172.21.15.28:0/1273269414 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68e4032900 con 0x7f68f4109a10 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.466+0000 7f68f1d9b640 1 -- 172.21.15.28:0/1273269414 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e4030850 con 0x7f68f4109a10 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 -- 172.21.15.28:0/1273269414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 msgr2=0x7f68f4109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/1273269414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009820 tx=0x7f68e402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 -- 172.21.15.28:0/1273269414 shutdown_connections 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/1273269414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f4109e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 -- 172.21.15.28:0/1273269414 >> 172.21.15.28:0/1273269414 conn(0x7f68f407d030 msgr2=0x7f68f407d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 -- 172.21.15.28:0/1273269414 shutdown_connections 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.467+0000 7f68f8ef8640 1 -- 172.21.15.28:0/1273269414 wait complete. 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f8ef8640 1 Processor -- start 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f8ef8640 1 -- start start 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f8ef8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f8ef8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410df70 con 0x7f68f4109a10 2024-09-19T09:23:17.766 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46568/0 (socket says 172.21.15.28:46568) 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f259c640 1 -- 172.21.15.28:0/3973959745 learned_addr learned my addr 172.21.15.28:0/3973959745 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.468+0000 7f68f259c640 1 -- 172.21.15.28:0/3973959745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f419df90 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68f259c640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f68e40097f0 tx=0x7f68e4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e4042070 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68e4032ca0 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f419e220 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e4037040 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.469+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41a0f10 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f68e4045070 con 0x7f68f4109a10 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68eb7fe640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 0x7f68e003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.767 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f68e406c0c0 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68ebfff640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 0x7f68e003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f410e0f0 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.470+0000 7f68ebfff640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 0x7f68e003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098c0 tx=0x7f68dc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.475+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68e403ccd0 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:16.697+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f68f41a13d0 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.190+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f68e403c7b0 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.191+0000 7f68eb7fe640 1 -- 172.21.15.28:0/3973959745 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f68e403da60 con 0x7f68f4109a10 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 msgr2=0x7f68e003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 0x7f68e003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098c0 tx=0x7f68dc006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 msgr2=0x7f68f419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f68e40097f0 tx=0x7f68e4030ec0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 shutdown_connections 2024-09-19T09:23:17.768 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f68e0039160 0x7f68e003b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.769 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 --2- 172.21.15.28:0/3973959745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f4109a10 0x7f68f419da50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.769 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 >> 172.21.15.28:0/3973959745 conn(0x7f68f407d030 msgr2=0x7f68f4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.769 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 shutdown_connections 2024-09-19T09:23:17.769 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:17.193+0000 7f68f8ef8640 1 -- 172.21.15.28:0/3973959745 wait complete. 2024-09-19T09:23:18.013 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.779+0000 7fd52a8ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:18.013 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:17 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:17.928+0000 7fd52a8ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:23:18.388 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:18 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3973959745' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T09:23:18.389 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:18 smithi028 ceph-mon[30972]: mgrmap e5: x(active, since 5s) 2024-09-19T09:23:18.389 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:18 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:18.269+0000 7fd52a8ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:23:18.389 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:18 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:18.333+0000 7fd52a8ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:23:18.705 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:18 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:18.531+0000 7fd52a8ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f5490f7f640 1 Processor -- start 2024-09-19T09:23:18.998 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f5490f7f640 1 -- start start 2024-09-19T09:23:18.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f5490f7f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:18.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f5490f7f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f548c07be00 con 0x7f548c106070 2024-09-19T09:23:18.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f548a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:18.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f548a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46598/0 (socket says 172.21.15.28:46598) 2024-09-19T09:23:18.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.180+0000 7f548a59c640 1 -- 172.21.15.28:0/3606541782 learned_addr learned my addr 172.21.15.28:0/3606541782 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.181+0000 7f548a59c640 1 -- 172.21.15.28:0/3606541782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c1069b0 con 0x7f548c106070 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.181+0000 7f548a59c640 1 --2- 172.21.15.28:0/3606541782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f547c009820 tx=0x7f547c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240b9b735be77083 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.181+0000 7f548959a640 1 -- 172.21.15.28:0/3606541782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f547c037070 con 0x7f548c106070 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f548959a640 1 -- 172.21.15.28:0/3606541782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f547c032900 con 0x7f548c106070 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f548959a640 1 -- 172.21.15.28:0/3606541782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f547c030850 con 0x7f548c106070 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 -- 172.21.15.28:0/3606541782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 msgr2=0x7f548c106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:19.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 --2- 172.21.15.28:0/3606541782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f547c009820 tx=0x7f547c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 -- 172.21.15.28:0/3606541782 shutdown_connections 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 --2- 172.21.15.28:0/3606541782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c106470 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 -- 172.21.15.28:0/3606541782 >> 172.21.15.28:0/3606541782 conn(0x7f548c101e60 msgr2=0x7f548c104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 -- 172.21.15.28:0/3606541782 shutdown_connections 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.182+0000 7f5490f7f640 1 -- 172.21.15.28:0/3606541782 wait complete. 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f5490f7f640 1 Processor -- start 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f5490f7f640 1 -- start start 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f5490f7f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f5490f7f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f548c107ab0 con 0x7f548c106070 2024-09-19T09:23:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f548a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f548a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46610/0 (socket says 172.21.15.28:46610) 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.183+0000 7f548a59c640 1 -- 172.21.15.28:0/1330333440 learned_addr learned my addr 172.21.15.28:0/1330333440 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f548a59c640 1 -- 172.21.15.28:0/1330333440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c19def0 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f548a59c640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097f0 tx=0x7f547c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f547c037040 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f547c040070 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f547c03a390 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c19e180 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.184+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c19e5a0 con 0x7f548c106070 2024-09-19T09:23:19.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f547c03a630 con 0x7f548c106070 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c106470 con 0x7f548c106070 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f547b7fe640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 0x7f547403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f547c06c460 con 0x7f548c106070 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f5489d9b640 1 -- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 msgr2=0x7f547403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.185+0000 7f5489d9b640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 0x7f547403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.194+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f547c04a200 con 0x7f548c106070 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.387+0000 7f5489d9b640 1 -- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 msgr2=0x7f547403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:19.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.387+0000 7f5489d9b640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 0x7f547403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.419+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f548c062df0 con 0x7f548c106070 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.420+0000 7f547b7fe640 1 -- 172.21.15.28:0/1330333440 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f547c02ee80 con 0x7f548c106070 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 msgr2=0x7f547403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 0x7f547403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 msgr2=0x7f548c19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097f0 tx=0x7f547c030d80 comp rx=0 tx=0).stop 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 shutdown_connections 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f54740391b0 0x7f547403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 --2- 172.21.15.28:0/1330333440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f548c106070 0x7f548c19d9b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:19.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 >> 172.21.15.28:0/1330333440 conn(0x7f548c101e60 msgr2=0x7f548c194670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:19.005 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 shutdown_connections 2024-09-19T09:23:19.005 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:18.422+0000 7f5490f7f640 1 -- 172.21.15.28:0/1330333440 wait complete. 2024-09-19T09:23:19.005 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for the mgr to restart... 2024-09-19T09:23:19.005 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr epoch 5... 2024-09-19T09:23:19.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:19 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1330333440' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T09:23:19.580 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:19 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:19.468+0000 7fd52a8ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:23:19.581 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:19 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:19.538+0000 7fd52a8ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:23:20.556 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.284+0000 7fd52a8ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:23:20.557 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.404+0000 7fd52a8ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:23:20.845 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.553+0000 7fd52a8ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:20.845 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.617+0000 7fd52a8ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:23:20.845 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.772+0000 7fd52a8ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:23:20.845 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.843+0000 7fd52a8ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:23:21.140 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:20 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:20.905+0000 7fd52a8ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:23:21.140 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.070+0000 7fd52a8ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:23:21.140 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.138+0000 7fd52a8ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:23:21.418 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.207+0000 7fd52a8ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:23:21.418 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.293+0000 7fd52a8ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:21.418 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.355+0000 7fd52a8ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:23:21.418 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:21.416+0000 7fd52a8ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:23:22.513 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:22.241+0000 7fd52a8ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: Active manager daemon x restarted 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: Activating manager daemon x 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: mgrmap e6: x(active, starting, since 0.00312625s) 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:22.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:22.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:22.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:22.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:23:22.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:22 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:23:23.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:23 smithi028 ceph-mon[30972]: Found migration_current of "None". Setting to last migration. 2024-09-19T09:23:23.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:23 smithi028 ceph-mon[30972]: mgrmap e7: x(active, since 1.00547s) 2024-09-19T09:23:23.881 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:23.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T09:23:23.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T09:23:23.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:23.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.372+0000 7f28d68e2640 1 Processor -- start 2024-09-19T09:23:23.882 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.372+0000 7f28d68e2640 1 -- start start 2024-09-19T09:23:23.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28d68e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:23.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28d68e2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0107090 con 0x7f28d01066c0 2024-09-19T09:23:23.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:23.883 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46614/0 (socket says 172.21.15.28:46614) 2024-09-19T09:23:23.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28cffff640 1 -- 172.21.15.28:0/3904009755 learned_addr learned my addr 172.21.15.28:0/3904009755 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:23.884 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.373+0000 7f28cffff640 1 -- 172.21.15.28:0/3904009755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d01078c0 con 0x7f28d01066c0 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.374+0000 7f28cffff640 1 --2- 172.21.15.28:0/3904009755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009820 tx=0x7f28c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afb5db32d9cb030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.374+0000 7f28cf7fe640 1 -- 172.21.15.28:0/3904009755 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0037070 con 0x7f28d01066c0 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.374+0000 7f28cf7fe640 1 -- 172.21.15.28:0/3904009755 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f28c0032900 con 0x7f28d01066c0 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.374+0000 7f28cf7fe640 1 -- 172.21.15.28:0/3904009755 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0030850 con 0x7f28d01066c0 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- 172.21.15.28:0/3904009755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 msgr2=0x7f28d0106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3904009755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009820 tx=0x7f28c002a270 comp rx=0 tx=0).stop 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- 172.21.15.28:0/3904009755 shutdown_connections 2024-09-19T09:23:23.885 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3904009755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d0106ac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- 172.21.15.28:0/3904009755 >> 172.21.15.28:0/3904009755 conn(0x7f28d0101e90 msgr2=0x7f28d01042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- 172.21.15.28:0/3904009755 shutdown_connections 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- 172.21.15.28:0/3904009755 wait complete. 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 Processor -- start 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.375+0000 7f28d68e2640 1 -- start start 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28d68e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28d68e2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28d010ac70 con 0x7f28d01066c0 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46626/0 (socket says 172.21.15.28:46626) 2024-09-19T09:23:23.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cffff640 1 -- 172.21.15.28:0/3151795590 learned_addr learned my addr 172.21.15.28:0/3151795590 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cffff640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019bdc0 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cffff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009940 tx=0x7f28c00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0042070 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f28c0032be0 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.376+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019c050 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.377+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0037040 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.377+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d019ed40 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.377+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f28c0045070 con 0x7f28d01066c0 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.378+0000 7f28cdffb640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:23.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.378+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f28d0106b40 con 0x7f28bc039200 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.378+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f28c006c100 con 0x7f28d01066c0 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.378+0000 7f28c7fff640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 msgr2=0x7f28bc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.378+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.579+0000 7f28c7fff640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 msgr2=0x7f28bc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.579+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.980+0000 7f28c7fff640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 msgr2=0x7f28bc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:19.980+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:20.781+0000 7f28c7fff640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 msgr2=0x7f28bc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/1042365655 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:20.781+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T09:23:23.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:22.244+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f28c006ac80 con 0x7f28d01066c0 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:22.245+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 msgr2=0x7f28bc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:22.245+0000 7f28cdffb640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.245+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f28c0042d00 con 0x7f28d01066c0 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.246+0000 7f28cdffb640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 0x7f28bc03ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.246+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f28d0106b40 con 0x7f28bc03c940 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.246+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 0x7f28bc03ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.247+0000 7f28c7fff640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 0x7f28bc03ed30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f28b8003980 tx=0x7f28b80092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.248+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f28d0106b40 con 0x7f28bc03c940 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f28d010adf0 con 0x7f28bc03c940 2024-09-19T09:23:23.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28cdffb640 1 -- 172.21.15.28:0/3151795590 <== mgr.14118 v2:172.21.15.28:6800/3407590911 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f28d010adf0 con 0x7f28bc03c940 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 msgr2=0x7f28bc03ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 0x7f28bc03ed30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f28b8003980 tx=0x7f28b80092b0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 msgr2=0x7f28d019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009940 tx=0x7f28c00326e0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 shutdown_connections 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/1042365655,v1:172.21.15.28:6801/1042365655] conn(0x7f28bc039200 0x7f28bc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f28bc03c940 0x7f28bc03ed30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 --2- 172.21.15.28:0/3151795590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28d01066c0 0x7f28d019b880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 >> 172.21.15.28:0/3151795590 conn(0x7f28d0101e90 msgr2=0x7f28d0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 shutdown_connections 2024-09-19T09:23:23.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:23.255+0000 7f28d68e2640 1 -- 172.21.15.28:0/3151795590 wait complete. 2024-09-19T09:23:23.891 INFO:teuthology.orchestra.run.smithi028.stdout:mgr epoch 5 is available 2024-09-19T09:23:23.891 INFO:teuthology.orchestra.run.smithi028.stdout:Setting orchestrator backend to cephadm... 2024-09-19T09:23:24.420 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:24 smithi028 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T09:23:24.420 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:24 smithi028 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T09:23:25.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea7df6640 1 Processor -- start 2024-09-19T09:23:25.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea7df6640 1 -- start start 2024-09-19T09:23:25.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea7df6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.041 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea7df6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea0107080 con 0x7f0ea01066b0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea5b92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea5b92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38428/0 (socket says 172.21.15.28:38428) 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.211+0000 7f0ea5b92640 1 -- 172.21.15.28:0/3186083185 learned_addr learned my addr 172.21.15.28:0/3186083185 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.212+0000 7f0ea5b92640 1 -- 172.21.15.28:0/3186083185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea01078b0 con 0x7f0ea01066b0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.212+0000 7f0ea5b92640 1 --2- 172.21.15.28:0/3186083185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009820 tx=0x7f0e9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d08b08736f97fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.212+0000 7f0ea4b90640 1 -- 172.21.15.28:0/3186083185 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c037070 con 0x7f0ea01066b0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.212+0000 7f0ea4b90640 1 -- 172.21.15.28:0/3186083185 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e9c032900 con 0x7f0ea01066b0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.212+0000 7f0ea4b90640 1 -- 172.21.15.28:0/3186083185 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c030850 con 0x7f0ea01066b0 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 -- 172.21.15.28:0/3186083185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 msgr2=0x7f0ea0106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/3186083185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009820 tx=0x7f0e9c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 -- 172.21.15.28:0/3186083185 shutdown_connections 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/3186083185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea0106ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 -- 172.21.15.28:0/3186083185 >> 172.21.15.28:0/3186083185 conn(0x7f0ea0101e60 msgr2=0x7f0ea0104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 -- 172.21.15.28:0/3186083185 shutdown_connections 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.213+0000 7f0ea7df6640 1 -- 172.21.15.28:0/3186083185 wait complete. 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea7df6640 1 Processor -- start 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea7df6640 1 -- start start 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea7df6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea7df6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea010ac60 con 0x7f0ea01066b0 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea5b92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea5b92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38440/0 (socket says 172.21.15.28:38440) 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea5b92640 1 -- 172.21.15.28:0/2069375685 learned_addr learned my addr 172.21.15.28:0/2069375685 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.214+0000 7f0ea5b92640 1 -- 172.21.15.28:0/2069375685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea019bd60 con 0x7f0ea01066b0 2024-09-19T09:23:25.043 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0ea5b92640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009940 tx=0x7f0e9c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c042070 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e9c0308c0 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea019bff0 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c037040 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.215+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea019ece0 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.216+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f0e9c045070 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.216+0000 7f0e96ffd640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 0x7f0e8c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.216+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea010ade0 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.216+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e9c06c290 con 0x7f0ea01066b0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.216+0000 7f0ea5391640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 0x7f0e8c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.217+0000 7f0ea5391640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 0x7f0e8c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e900098c0 tx=0x7f0e90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:25.044 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.225+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e9c03add0 con 0x7f0ea01066b0 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.297+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0e9c03c760 con 0x7f0ea01066b0 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.418+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0ea002da90 con 0x7f0e8c0390f0 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.424+0000 7f0e96ffd640 1 -- 172.21.15.28:0/2069375685 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ea002da90 con 0x7f0e8c0390f0 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 msgr2=0x7f0e8c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 0x7f0e8c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e900098c0 tx=0x7f0e90006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 msgr2=0x7f0ea019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009940 tx=0x7f0e9c032ce0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 shutdown_connections 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0e8c0390f0 0x7f0e8c03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 --2- 172.21.15.28:0/2069375685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ea01066b0 0x7f0ea019b820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 >> 172.21.15.28:0/2069375685 conn(0x7f0ea0101e60 msgr2=0x7f0ea0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 shutdown_connections 2024-09-19T09:23:25.045 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:24.426+0000 7f0ea7df6640 1 -- 172.21.15.28:0/2069375685 wait complete. 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: mgrmap e8: x(active, since 2s) 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:25.663 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:25 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:26.241 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T09:23:26.241 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb7724ec640 1 Processor -- start 2024-09-19T09:23:26.241 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb7724ec640 1 -- start start 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb7724ec640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb7724ec640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c107090 con 0x7fb76c1066c0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb76bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb76bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38446/0 (socket says 172.21.15.28:38446) 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.474+0000 7fb76bfff640 1 -- 172.21.15.28:0/224862715 learned_addr learned my addr 172.21.15.28:0/224862715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.475+0000 7fb76bfff640 1 -- 172.21.15.28:0/224862715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c1078c0 con 0x7fb76c1066c0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.475+0000 7fb76bfff640 1 --2- 172.21.15.28:0/224862715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009820 tx=0x7fb75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5157f2f084bea50e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb76affd640 1 -- 172.21.15.28:0/224862715 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c037070 con 0x7fb76c1066c0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb76affd640 1 -- 172.21.15.28:0/224862715 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c032900 con 0x7fb76c1066c0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb76affd640 1 -- 172.21.15.28:0/224862715 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c030850 con 0x7fb76c1066c0 2024-09-19T09:23:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 -- 172.21.15.28:0/224862715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 msgr2=0x7fb76c106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 --2- 172.21.15.28:0/224862715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009820 tx=0x7fb75c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 -- 172.21.15.28:0/224862715 shutdown_connections 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 --2- 172.21.15.28:0/224862715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c106ac0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 -- 172.21.15.28:0/224862715 >> 172.21.15.28:0/224862715 conn(0x7fb76c101e90 msgr2=0x7fb76c1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 -- 172.21.15.28:0/224862715 shutdown_connections 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.476+0000 7fb7724ec640 1 -- 172.21.15.28:0/224862715 wait complete. 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 Processor -- start 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 -- start start 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c07b100 con 0x7fb76c1066c0 2024-09-19T09:23:26.243 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb76bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb76bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38452/0 (socket says 172.21.15.28:38452) 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb76bfff640 1 -- 172.21.15.28:0/2007430588 learned_addr learned my addr 172.21.15.28:0/2007430588 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb76bfff640 1 -- 172.21.15.28:0/2007430588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19bd80 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb76bfff640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097f0 tx=0x7fb75c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c042070 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c032ca0 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19c010 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c19ed00 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.478+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c037040 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.480+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb75c045070 con 0x7fb76c1066c0 2024-09-19T09:23:26.244 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.480+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c106ac0 con 0x7fb76c1066c0 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.480+0000 7fb7697fa640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 0x7fb75403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.480+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb75c06c1a0 con 0x7fb76c1066c0 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.481+0000 7fb76b7fe640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 0x7fb75403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.481+0000 7fb76b7fe640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 0x7fb75403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098c0 tx=0x7fb760006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.484+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb75c03d490 con 0x7fb76c1066c0 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.662+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb76c02dab0 con 0x7fb754039200 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.663+0000 7fb7697fa640 1 -- 172.21.15.28:0/2007430588 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb76c02dab0 con 0x7fb754039200 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 msgr2=0x7fb75403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 0x7fb75403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098c0 tx=0x7fb760006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:26.245 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 msgr2=0x7fb76c19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097f0 tx=0x7fb75c030ec0 comp rx=0 tx=0).stop 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 shutdown_connections 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb754039200 0x7fb75403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.665+0000 7fb7724ec640 1 --2- 172.21.15.28:0/2007430588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76c1066c0 0x7fb76c19b840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.666+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 >> 172.21.15.28:0/2007430588 conn(0x7fb76c101e90 msgr2=0x7fb76c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.666+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 shutdown_connections 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:25.666+0000 7fb7724ec640 1 -- 172.21.15.28:0/2007430588 wait complete. 2024-09-19T09:23:26.246 INFO:teuthology.orchestra.run.smithi028.stdout:Generating ssh key... 2024-09-19T09:23:26.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-mon[30972]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:26.648+0000 7fd497413640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:09:23:26] ENGINE Error in HTTPServer.serve 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: Traceback (most recent call last): 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._connections.run(self.expiration_interval) 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._run(expiration_interval) 2024-09-19T09:23:26.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: s = self.context.wrap_socket( 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: return self.sslsocket_class._create( 2024-09-19T09:23:26.826 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:23:26.827 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self.do_handshake() 2024-09-19T09:23:26.827 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:23:26.827 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._sslobj.do_handshake() 2024-09-19T09:23:26.827 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:23:26.827 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:26] ENGINE Bus STARTING 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:26] ENGINE Serving on https://172.21.15.28:7150 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:26] ENGINE Error in HTTPServer.serve 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: Traceback (most recent call last): 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._run(expiration_interval) 2024-09-19T09:23:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: s = self.context.wrap_socket( 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: return self.sslsocket_class._create( 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: self.do_handshake() 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._sslobj.do_handshake() 2024-09-19T09:23:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:26] ENGINE Serving on http://172.21.15.28:8765 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:26] ENGINE Bus STARTED 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-mon[30972]: Generating ssh key... 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: Generating public/private rsa key pair. 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: Your identification has been saved in /tmp/tmprjm51w5d/key 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: Your public key has been saved in /tmp/tmprjm51w5d/key.pub 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: The key fingerprint is: 2024-09-19T09:23:27.766 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: SHA256:CiUKJ/U7YN2Ii5mCJoLwEQuriqH06DgrKKPUDBZPkzs ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: The key's randomart image is: 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: +---[RSA 3072]----+ 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |. o | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: | + *.o | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |=.B+= o | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |=X+=o+ | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |%=+E+ S | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |@.* .o . | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |=o + . | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |X | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: |B= | 2024-09-19T09:23:27.767 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: +----[SHA256]-----+ 2024-09-19T09:23:28.109 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.638+0000 7f566a06e640 1 Processor -- start 2024-09-19T09:23:28.114 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.638+0000 7f566a06e640 1 -- start start 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f566a06e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f566a06e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5664107080 con 0x7f56641066b0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f56637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f56637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38458/0 (socket says 172.21.15.28:38458) 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f56637fe640 1 -- 172.21.15.28:0/2215056232 learned_addr learned my addr 172.21.15.28:0/2215056232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.639+0000 7f56637fe640 1 -- 172.21.15.28:0/2215056232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641078b0 con 0x7f56641066b0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.640+0000 7f56637fe640 1 --2- 172.21.15.28:0/2215056232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f565c009e80 tx=0x7f565c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a086ef2d7367dbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.640+0000 7f56627fc640 1 -- 172.21.15.28:0/2215056232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c036070 con 0x7f56641066b0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.640+0000 7f56627fc640 1 -- 172.21.15.28:0/2215056232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f565c0318c0 con 0x7f56641066b0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.640+0000 7f56627fc640 1 -- 172.21.15.28:0/2215056232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c007ab0 con 0x7f56641066b0 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- 172.21.15.28:0/2215056232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 msgr2=0x7f5664106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 --2- 172.21.15.28:0/2215056232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f565c009e80 tx=0x7f565c02a7a0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.115 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- 172.21.15.28:0/2215056232 shutdown_connections 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 --2- 172.21.15.28:0/2215056232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f5664106ab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- 172.21.15.28:0/2215056232 >> 172.21.15.28:0/2215056232 conn(0x7f5664101e60 msgr2=0x7f5664104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- 172.21.15.28:0/2215056232 shutdown_connections 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- 172.21.15.28:0/2215056232 wait complete. 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 Processor -- start 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.641+0000 7f566a06e640 1 -- start start 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f566a06e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f566a06e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f566410ac60 con 0x7f56641066b0 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f56637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f56637fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38472/0 (socket says 172.21.15.28:38472) 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f56637fe640 1 -- 172.21.15.28:0/1226726444 learned_addr learned my addr 172.21.15.28:0/1226726444 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:28.116 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f56637fe640 1 -- 172.21.15.28:0/1226726444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566419bd60 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f56637fe640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f565c02acc0 tx=0x7f565c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c036050 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f565c037040 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566419bff0 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.642+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c03d8c0 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.643+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566419ece0 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.643+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f565c044020 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.644+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566410ade0 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.644+0000 7f5660ff9640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 0x7f565003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.644+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f565c042020 con 0x7f56641066b0 2024-09-19T09:23:28.117 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.644+0000 7f5662ffd640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 0x7f565003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.644+0000 7f5662ffd640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 0x7f565003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f564c0098c0 tx=0x7f564c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.649+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f565c04b020 con 0x7f56641066b0 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:26.824+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f566402daa0 con 0x7f5650039200 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.519+0000 7f5660ff9640 1 -- 172.21.15.28:0/1226726444 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f566402daa0 con 0x7f5650039200 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 msgr2=0x7f565003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 0x7f565003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f564c0098c0 tx=0x7f564c006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 msgr2=0x7f566419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f565c02acc0 tx=0x7f565c031ca0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 shutdown_connections 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5650039200 0x7f565003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 --2- 172.21.15.28:0/1226726444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641066b0 0x7f566419b820 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:28.118 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 >> 172.21.15.28:0/1226726444 conn(0x7f5664101e60 msgr2=0x7f5664102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:28.119 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.521+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 shutdown_connections 2024-09-19T09:23:28.119 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:27.522+0000 7f566a06e640 1 -- 172.21.15.28:0/1226726444 wait complete. 2024-09-19T09:23:28.737 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:28 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:28.738 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:28 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:29.462 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:29.462 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.546+0000 7f26c1989640 1 Processor -- start 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.546+0000 7f26c1989640 1 -- start start 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.546+0000 7f26c1989640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.546+0000 7f26c1989640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc107080 con 0x7f26bc1066b0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.547+0000 7f26baffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.547+0000 7f26baffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38486/0 (socket says 172.21.15.28:38486) 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.547+0000 7f26baffd640 1 -- 172.21.15.28:0/1383291187 learned_addr learned my addr 172.21.15.28:0/1383291187 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.547+0000 7f26baffd640 1 -- 172.21.15.28:0/1383291187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc1078b0 con 0x7f26bc1066b0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.548+0000 7f26baffd640 1 --2- 172.21.15.28:0/1383291187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009820 tx=0x7f26ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1cd992f9ae7123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.548+0000 7f26b9ffb640 1 -- 172.21.15.28:0/1383291187 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ac037070 con 0x7f26bc1066b0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.548+0000 7f26b9ffb640 1 -- 172.21.15.28:0/1383291187 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26ac032900 con 0x7f26bc1066b0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.548+0000 7f26b9ffb640 1 -- 172.21.15.28:0/1383291187 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ac030850 con 0x7f26bc1066b0 2024-09-19T09:23:29.463 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 -- 172.21.15.28:0/1383291187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 msgr2=0x7f26bc106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 --2- 172.21.15.28:0/1383291187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009820 tx=0x7f26ac02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 -- 172.21.15.28:0/1383291187 shutdown_connections 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 --2- 172.21.15.28:0/1383291187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc106ab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 -- 172.21.15.28:0/1383291187 >> 172.21.15.28:0/1383291187 conn(0x7f26bc101e60 msgr2=0x7f26bc104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 -- 172.21.15.28:0/1383291187 shutdown_connections 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 -- 172.21.15.28:0/1383291187 wait complete. 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.549+0000 7f26c1989640 1 Processor -- start 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26c1989640 1 -- start start 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26c1989640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26c1989640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc10ac60 con 0x7f26bc1066b0 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26baffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26baffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38496/0 (socket says 172.21.15.28:38496) 2024-09-19T09:23:29.464 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26baffd640 1 -- 172.21.15.28:0/3240666697 learned_addr learned my addr 172.21.15.28:0/3240666697 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26baffd640 1 -- 172.21.15.28:0/3240666697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc191e20 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.550+0000 7f26baffd640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009940 tx=0x7f26ac032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ac042070 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26ac0308c0 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc190010 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ac037040 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc190550 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.551+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f26ac045070 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.552+0000 7f26c0987640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 0x7f26a403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.552+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26bc106b30 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.552+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26ac06c370 con 0x7f26bc1066b0 2024-09-19T09:23:29.465 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.552+0000 7f26ba7fc640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 0x7f26a403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.552+0000 7f26ba7fc640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 0x7f26a403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26b00098c0 tx=0x7f26b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.557+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f26ac03cdb0 con 0x7f26bc1066b0 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.737+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f26bc1030f0 con 0x7f26a40391b0 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.738+0000 7f26c0987640 1 -- 172.21.15.28:0/3240666697 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f26bc1030f0 con 0x7f26a40391b0 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 msgr2=0x7f26a403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 0x7f26a403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26b00098c0 tx=0x7f26b0006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 msgr2=0x7f26bc1918e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009940 tx=0x7f26ac032ce0 comp rx=0 tx=0).stop 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 shutdown_connections 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f26a40391b0 0x7f26a403b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 --2- 172.21.15.28:0/3240666697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26bc1066b0 0x7f26bc1918e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 >> 172.21.15.28:0/3240666697 conn(0x7f26bc101e60 msgr2=0x7f26bc1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:29.466 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 shutdown_connections 2024-09-19T09:23:29.467 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:28.741+0000 7f26c1989640 1 -- 172.21.15.28:0/3240666697 wait complete. 2024-09-19T09:23:29.467 INFO:teuthology.orchestra.run.smithi028.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:23:29.467 INFO:teuthology.orchestra.run.smithi028.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T09:23:29.467 INFO:teuthology.orchestra.run.smithi028.stdout:Adding host smithi028... 2024-09-19T09:23:29.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:29 smithi028 ceph-mon[30972]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:30.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:30 smithi028 ceph-mon[30972]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi028", "addr": "172.21.15.28", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:31.699 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:31 smithi028 ceph-mon[30972]: Deploying cephadm binary to smithi028 2024-09-19T09:23:34.477 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Added host 'smithi028' with addr '172.21.15.28' 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe55191e640 1 Processor -- start 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe55191e640 1 -- start start 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe55191e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe55191e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10a3e0 con 0x7fe54c109a10 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe54affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe54affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38504/0 (socket says 172.21.15.28:38504) 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.857+0000 7fe54affd640 1 -- 172.21.15.28:0/2976528242 learned_addr learned my addr 172.21.15.28:0/2976528242 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.858+0000 7fe54affd640 1 -- 172.21.15.28:0/2976528242 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c10abc0 con 0x7fe54c109a10 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.858+0000 7fe54affd640 1 --2- 172.21.15.28:0/2976528242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009820 tx=0x7fe53c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0ad89ef1fc5a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe54a7fc640 1 -- 172.21.15.28:0/2976528242 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe53c037070 con 0x7fe54c109a10 2024-09-19T09:23:34.478 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe54a7fc640 1 -- 172.21.15.28:0/2976528242 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe53c032900 con 0x7fe54c109a10 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe54a7fc640 1 -- 172.21.15.28:0/2976528242 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe53c030850 con 0x7fe54c109a10 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 -- 172.21.15.28:0/2976528242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 msgr2=0x7fe54c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 --2- 172.21.15.28:0/2976528242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009820 tx=0x7fe53c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 -- 172.21.15.28:0/2976528242 shutdown_connections 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 --2- 172.21.15.28:0/2976528242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c109e10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 -- 172.21.15.28:0/2976528242 >> 172.21.15.28:0/2976528242 conn(0x7fe54c07d030 msgr2=0x7fe54c07d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 -- 172.21.15.28:0/2976528242 shutdown_connections 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.859+0000 7fe55191e640 1 -- 172.21.15.28:0/2976528242 wait complete. 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe55191e640 1 Processor -- start 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe55191e640 1 -- start start 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe55191e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:34.479 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe55191e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10df70 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe54affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe54affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38518/0 (socket says 172.21.15.28:38518) 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.860+0000 7fe54affd640 1 -- 172.21.15.28:0/636953410 learned_addr learned my addr 172.21.15.28:0/636953410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe54affd640 1 -- 172.21.15.28:0/636953410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19df90 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe54affd640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0097f0 tx=0x7fe53c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe53c042070 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe53c032ca0 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c19e220 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe53c037040 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.861+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a0f10 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.862+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fe53c045070 con 0x7fe54c109a10 2024-09-19T09:23:34.480 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.862+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c10e0f0 con 0x7fe54c109a10 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.863+0000 7fe55091c640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 0x7fe53803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.863+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe53c06c370 con 0x7fe54c109a10 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.863+0000 7fe5427fc640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 0x7fe53803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.863+0000 7fe5427fc640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 0x7fe53803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5340098c0 tx=0x7fe534006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:29.867+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe53c03be10 con 0x7fe54c109a10 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:30.047+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi028", "addr": "172.21.15.28", "target": ["mon-mgr", ""]}) -- 0x7fe54c107c60 con 0x7fe5380391b0 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.958+0000 7fe55091c640 1 -- 172.21.15.28:0/636953410 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe54c107c60 con 0x7fe5380391b0 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 msgr2=0x7fe53803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 0x7fe53803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5340098c0 tx=0x7fe534006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 msgr2=0x7fe54c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0097f0 tx=0x7fe53c030ec0 comp rx=0 tx=0).stop 2024-09-19T09:23:34.481 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 shutdown_connections 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fe5380391b0 0x7fe53803b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 --2- 172.21.15.28:0/636953410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe54c109a10 0x7fe54c19da50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 >> 172.21.15.28:0/636953410 conn(0x7fe54c07d030 msgr2=0x7fe54c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 shutdown_connections 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:33.962+0000 7fe55191e640 1 -- 172.21.15.28:0/636953410 wait complete. 2024-09-19T09:23:34.482 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying unmanaged mon service... 2024-09-19T09:23:35.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:34 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:35.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:34 smithi028 ceph-mon[30972]: Added host smithi028 2024-09-19T09:23:35.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:34 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:35.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T09:23:35.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.890+0000 7fb07f457640 1 Processor -- start 2024-09-19T09:23:35.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07f457640 1 -- start start 2024-09-19T09:23:35.886 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07f457640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07f457640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781070f0 con 0x7fb078106720 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07d1f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07d1f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38054/0 (socket says 172.21.15.28:38054) 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.891+0000 7fb07d1f3640 1 -- 172.21.15.28:0/210133684 learned_addr learned my addr 172.21.15.28:0/210133684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.892+0000 7fb07d1f3640 1 -- 172.21.15.28:0/210133684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb078107920 con 0x7fb078106720 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.892+0000 7fb07d1f3640 1 --2- 172.21.15.28:0/210133684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb074009820 tx=0x7fb07402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e351c88d0f47db11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb06ffff640 1 -- 172.21.15.28:0/210133684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074037070 con 0x7fb078106720 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb06ffff640 1 -- 172.21.15.28:0/210133684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb074032900 con 0x7fb078106720 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb06ffff640 1 -- 172.21.15.28:0/210133684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074030850 con 0x7fb078106720 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 -- 172.21.15.28:0/210133684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 msgr2=0x7fb078106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 --2- 172.21.15.28:0/210133684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb074009820 tx=0x7fb07402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:35.887 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 -- 172.21.15.28:0/210133684 shutdown_connections 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 --2- 172.21.15.28:0/210133684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb078106b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 -- 172.21.15.28:0/210133684 >> 172.21.15.28:0/210133684 conn(0x7fb078101f10 msgr2=0x7fb078104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 -- 172.21.15.28:0/210133684 shutdown_connections 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.893+0000 7fb07f457640 1 -- 172.21.15.28:0/210133684 wait complete. 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07f457640 1 Processor -- start 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07f457640 1 -- start start 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07f457640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07f457640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810acd0 con 0x7fb078106720 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07d1f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07d1f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38062/0 (socket says 172.21.15.28:38062) 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.894+0000 7fb07d1f3640 1 -- 172.21.15.28:0/1580619559 learned_addr learned my addr 172.21.15.28:0/1580619559 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:35.888 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb07d1f3640 1 -- 172.21.15.28:0/1580619559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07819ac80 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb07d1f3640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb074009940 tx=0x7fb074032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074042070 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0740308c0 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07819af10 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074037040 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.895+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07819dc00 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.896+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb074045070 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.896+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07810ae50 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.896+0000 7fb06e7fc640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 0x7fb06403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.896+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb07406c4f0 con 0x7fb078106720 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.896+0000 7fb07c9f2640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 0x7fb06403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.897+0000 7fb07c9f2640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 0x7fb06403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0680098c0 tx=0x7fb068006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:35.889 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:34.901+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb074045e40 con 0x7fb078106720 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.115+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb078103170 con 0x7fb064039200 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.120+0000 7fb06e7fc640 1 -- 172.21.15.28:0/1580619559 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb078103170 con 0x7fb064039200 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 msgr2=0x7fb06403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 0x7fb06403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0680098c0 tx=0x7fb068006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 msgr2=0x7fb07819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb074009940 tx=0x7fb074032ce0 comp rx=0 tx=0).stop 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 shutdown_connections 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fb064039200 0x7fb06403b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 --2- 172.21.15.28:0/1580619559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb078106720 0x7fb07819a740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 >> 172.21.15.28:0/1580619559 conn(0x7fb078101f10 msgr2=0x7fb078102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 shutdown_connections 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:35.122+0000 7fb07f457640 1 -- 172.21.15.28:0/1580619559 wait complete. 2024-09-19T09:23:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:Deploying unmanaged mgr service... 2024-09-19T09:23:36.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:36 smithi028 ceph-mon[30972]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:36.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:36 smithi028 ceph-mon[30972]: Saving service mon spec with placement count:5 2024-09-19T09:23:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:36 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:37.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T09:23:37.874 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f566aad4640 1 Processor -- start 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f566aad4640 1 -- start start 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f566aad4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f566aad4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f566407be20 con 0x7f56641082d0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f5668870640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f5668870640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38076/0 (socket says 172.21.15.28:38076) 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.529+0000 7f5668870640 1 -- 172.21.15.28:0/4247756100 learned_addr learned my addr 172.21.15.28:0/4247756100 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.530+0000 7f5668870640 1 -- 172.21.15.28:0/4247756100 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566410ac00 con 0x7f56641082d0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.530+0000 7f5668870640 1 --2- 172.21.15.28:0/4247756100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5658009820 tx=0x7f565802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8fc8b71f495f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.530+0000 7f56637fe640 1 -- 172.21.15.28:0/4247756100 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5658037070 con 0x7f56641082d0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.530+0000 7f56637fe640 1 -- 172.21.15.28:0/4247756100 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5658032900 con 0x7f56641082d0 2024-09-19T09:23:37.875 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.530+0000 7f56637fe640 1 -- 172.21.15.28:0/4247756100 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5658030850 con 0x7f56641082d0 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 -- 172.21.15.28:0/4247756100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 msgr2=0x7f566410a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 --2- 172.21.15.28:0/4247756100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5658009820 tx=0x7f565802a270 comp rx=0 tx=0).stop 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 -- 172.21.15.28:0/4247756100 shutdown_connections 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 --2- 172.21.15.28:0/4247756100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f566410a6c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 -- 172.21.15.28:0/4247756100 >> 172.21.15.28:0/4247756100 conn(0x7f5664101f10 msgr2=0x7f5664104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 -- 172.21.15.28:0/4247756100 shutdown_connections 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.531+0000 7f566aad4640 1 -- 172.21.15.28:0/4247756100 wait complete. 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f566aad4640 1 Processor -- start 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f566aad4640 1 -- start start 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f566aad4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:37.876 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f566aad4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f566410bc40 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f5668870640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f5668870640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38090/0 (socket says 172.21.15.28:38090) 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f5668870640 1 -- 172.21.15.28:0/2305139504 learned_addr learned my addr 172.21.15.28:0/2305139504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.532+0000 7f5668870640 1 -- 172.21.15.28:0/2305139504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5664108090 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f5668870640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5658032ce0 tx=0x7f5658032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5658042070 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56580308c0 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5664106280 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5658037040 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.533+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641067c0 con 0x7f56641082d0 2024-09-19T09:23:37.877 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.534+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5658045070 con 0x7f56641082d0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.534+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5664108730 con 0x7f56641082d0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.534+0000 7f5661ffb640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 0x7f565403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.534+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f565806c2b0 con 0x7f56641082d0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.534+0000 7f5663fff640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 0x7f565403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.535+0000 7f5663fff640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 0x7f565403b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56500098c0 tx=0x7f5650006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.539+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f565804b170 con 0x7f56641082d0 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.730+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5664107070 con 0x7f5654039200 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.734+0000 7f5661ffb640 1 -- 172.21.15.28:0/2305139504 <== mgr.14118 v2:172.21.15.28:6800/3407590911 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5664107070 con 0x7f5654039200 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 msgr2=0x7f565403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 0x7f565403b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56500098c0 tx=0x7f5650006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 msgr2=0x7f5664107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5658032ce0 tx=0x7f5658032d10 comp rx=0 tx=0).stop 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 shutdown_connections 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f5654039200 0x7f565403b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 --2- 172.21.15.28:0/2305139504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56641082d0 0x7f5664107b50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 >> 172.21.15.28:0/2305139504 conn(0x7f5664101f10 msgr2=0x7f5664102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 shutdown_connections 2024-09-19T09:23:37.879 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:36.737+0000 7f566aad4640 1 -- 172.21.15.28:0/2305139504 wait complete. 2024-09-19T09:23:37.995 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:37 smithi028 ceph-mon[30972]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:37.995 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:37 smithi028 ceph-mon[30972]: Saving service mgr spec with placement count:2 2024-09-19T09:23:37.995 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:37 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:37.996 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:37 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:39.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:38 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:39.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:38 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/270747043' entity='client.admin' 2024-09-19T09:23:39.717 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.320+0000 7f0f7414f640 1 Processor -- start 2024-09-19T09:23:39.717 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.320+0000 7f0f7414f640 1 -- start start 2024-09-19T09:23:39.717 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.320+0000 7f0f7414f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:39.717 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.320+0000 7f0f7414f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1064b0 con 0x7f0f6c105ae0 2024-09-19T09:23:39.717 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.321+0000 7f0f71eeb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.321+0000 7f0f71eeb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38100/0 (socket says 172.21.15.28:38100) 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.321+0000 7f0f71eeb640 1 -- 172.21.15.28:0/402105021 learned_addr learned my addr 172.21.15.28:0/402105021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.321+0000 7f0f71eeb640 1 -- 172.21.15.28:0/402105021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c108ac0 con 0x7f0f6c105ae0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f71eeb640 1 --2- 172.21.15.28:0/402105021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009820 tx=0x7f0f6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661ce018cb070191 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f70ee9640 1 -- 172.21.15.28:0/402105021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f68037070 con 0x7f0f6c105ae0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f70ee9640 1 -- 172.21.15.28:0/402105021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f68032920 con 0x7f0f6c105ae0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f70ee9640 1 -- 172.21.15.28:0/402105021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f68030810 con 0x7f0f6c105ae0 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f7414f640 1 -- 172.21.15.28:0/402105021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 msgr2=0x7f0f6c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f7414f640 1 --2- 172.21.15.28:0/402105021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009820 tx=0x7f0f6802a270 comp rx=0 tx=0).stop 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f7414f640 1 -- 172.21.15.28:0/402105021 shutdown_connections 2024-09-19T09:23:39.718 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f7414f640 1 --2- 172.21.15.28:0/402105021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c105ee0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.322+0000 7f0f7414f640 1 -- 172.21.15.28:0/402105021 >> 172.21.15.28:0/402105021 conn(0x7f0f6c1016b0 msgr2=0x7f0f6c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 -- 172.21.15.28:0/402105021 shutdown_connections 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 -- 172.21.15.28:0/402105021 wait complete. 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 Processor -- start 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 -- start start 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f7414f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c10bcf0 con 0x7f0f6c105ae0 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f71eeb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f71eeb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38114/0 (socket says 172.21.15.28:38114) 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.323+0000 7f0f71eeb640 1 -- 172.21.15.28:0/270747043 learned_addr learned my addr 172.21.15.28:0/270747043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f71eeb640 1 -- 172.21.15.28:0/270747043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c19bdf0 con 0x7f0f6c105ae0 2024-09-19T09:23:39.719 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f71eeb640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f68030560 tx=0x7f0f68032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f68037040 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f68040070 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c19c080 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f6803a6d0 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.324+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c19ed70 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.325+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0f6803a870 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.325+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c105f60 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.325+0000 7f0f62ffd640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 0x7f0f5803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.325+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f6806c950 con 0x7f0f6c105ae0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.325+0000 7f0f716ea640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 0x7f0f5803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.326+0000 7f0f716ea640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 0x7f0f5803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098c0 tx=0x7f0f5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:39.720 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.330+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0f68032920 con 0x7f0f6c105ae0 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.515+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0f6c19f370 con 0x7f0f6c105ae0 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.517+0000 7f0f62ffd640 1 -- 172.21.15.28:0/270747043 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0f68045980 con 0x7f0f6c105ae0 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 msgr2=0x7f0f5803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 0x7f0f5803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098c0 tx=0x7f0f5c006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 msgr2=0x7f0f6c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f68030560 tx=0x7f0f68032670 comp rx=0 tx=0).stop 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 shutdown_connections 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f0f58039200 0x7f0f5803b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 --2- 172.21.15.28:0/270747043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f6c105ae0 0x7f0f6c19b8b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 >> 172.21.15.28:0/270747043 conn(0x7f0f6c1016b0 msgr2=0x7f0f6c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 shutdown_connections 2024-09-19T09:23:39.721 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:38.520+0000 7f0f7414f640 1 -- 172.21.15.28:0/270747043 wait complete. 2024-09-19T09:23:40.342 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:40 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:40.934 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.140+0000 7fa43766a640 1 Processor -- start 2024-09-19T09:23:40.934 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa43766a640 1 -- start start 2024-09-19T09:23:40.934 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa43766a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:40.934 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa43766a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa430107080 con 0x7fa4301066b0 2024-09-19T09:23:40.934 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa435406640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:40.935 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa435406640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38126/0 (socket says 172.21.15.28:38126) 2024-09-19T09:23:40.935 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa435406640 1 -- 172.21.15.28:0/435238993 learned_addr learned my addr 172.21.15.28:0/435238993 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:40.935 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.141+0000 7fa435406640 1 -- 172.21.15.28:0/435238993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4301078b0 con 0x7fa4301066b0 2024-09-19T09:23:40.935 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.142+0000 7fa435406640 1 --2- 172.21.15.28:0/435238993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009820 tx=0x7fa42c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25ebe7fcae1f947 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:40.935 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.142+0000 7fa427fff640 1 -- 172.21.15.28:0/435238993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42c037070 con 0x7fa4301066b0 2024-09-19T09:23:40.936 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.142+0000 7fa427fff640 1 -- 172.21.15.28:0/435238993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c032920 con 0x7fa4301066b0 2024-09-19T09:23:40.936 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.142+0000 7fa427fff640 1 -- 172.21.15.28:0/435238993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42c030810 con 0x7fa4301066b0 2024-09-19T09:23:40.936 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 -- 172.21.15.28:0/435238993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 msgr2=0x7fa430106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:40.936 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 --2- 172.21.15.28:0/435238993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009820 tx=0x7fa42c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:40.936 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 -- 172.21.15.28:0/435238993 shutdown_connections 2024-09-19T09:23:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 --2- 172.21.15.28:0/435238993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa430106ab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 -- 172.21.15.28:0/435238993 >> 172.21.15.28:0/435238993 conn(0x7fa430101e60 msgr2=0x7fa430104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 -- 172.21.15.28:0/435238993 shutdown_connections 2024-09-19T09:23:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.143+0000 7fa43766a640 1 -- 172.21.15.28:0/435238993 wait complete. 2024-09-19T09:23:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa43766a640 1 Processor -- start 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa43766a640 1 -- start start 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa43766a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa43766a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa43010ac60 con 0x7fa4301066b0 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa435406640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa435406640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38132/0 (socket says 172.21.15.28:38132) 2024-09-19T09:23:40.938 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa435406640 1 -- 172.21.15.28:0/1631186065 learned_addr learned my addr 172.21.15.28:0/1631186065 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:40.939 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa435406640 1 -- 172.21.15.28:0/1631186065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43019c380 con 0x7fa4301066b0 2024-09-19T09:23:40.939 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa435406640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0097f0 tx=0x7fa42c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:40.939 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42c037040 con 0x7fa4301066b0 2024-09-19T09:23:40.939 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4301a0990 con 0x7fa4301066b0 2024-09-19T09:23:40.939 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c030cc0 con 0x7fa4301066b0 2024-09-19T09:23:40.940 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.144+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42c03a640 con 0x7fa4301066b0 2024-09-19T09:23:40.940 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.145+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4301a0d30 con 0x7fa4301066b0 2024-09-19T09:23:40.940 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.145+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa42c03b3f0 con 0x7fa4301066b0 2024-09-19T09:23:40.940 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.145+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa430106b30 con 0x7fa4301066b0 2024-09-19T09:23:40.940 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.146+0000 7fa4267fc640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 0x7fa41c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:40.941 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.146+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa42c06c7d0 con 0x7fa4301066b0 2024-09-19T09:23:40.941 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.146+0000 7fa434c05640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 0x7fa41c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:40.941 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.146+0000 7fa434c05640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 0x7fa41c03b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa4200098c0 tx=0x7fa420006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:40.941 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.150+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa42c0328c0 con 0x7fa4301066b0 2024-09-19T09:23:40.941 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.340+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa4301a1240 con 0x7fa4301066b0 2024-09-19T09:23:40.942 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.343+0000 7fa4267fc640 1 -- 172.21.15.28:0/1631186065 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa42c042dc0 con 0x7fa4301066b0 2024-09-19T09:23:40.942 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 msgr2=0x7fa41c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:40.942 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 0x7fa41c03b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa4200098c0 tx=0x7fa420006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:40.942 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 msgr2=0x7fa43007ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:40.943 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0097f0 tx=0x7fa42c0327b0 comp rx=0 tx=0).stop 2024-09-19T09:23:40.943 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 shutdown_connections 2024-09-19T09:23:40.943 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fa41c0391b0 0x7fa41c03b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:40.943 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 --2- 172.21.15.28:0/1631186065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4301066b0 0x7fa43007ba00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:40.943 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.345+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 >> 172.21.15.28:0/1631186065 conn(0x7fa430101e60 msgr2=0x7fa430102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:40.944 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.346+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 shutdown_connections 2024-09-19T09:23:40.944 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:40.346+0000 7fa43766a640 1 -- 172.21.15.28:0/1631186065 wait complete. 2024-09-19T09:23:40.944 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling the dashboard module... 2024-09-19T09:23:41.305 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:41 smithi028 ceph-mon[30972]: from='mgr.14118 172.21.15.28:0/1245882711' entity='mgr.x' 2024-09-19T09:23:41.305 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:41 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1631186065' entity='client.admin' 2024-09-19T09:23:42.403 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2696658328' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T09:23:42.404 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ignoring --setuser ceph since I am not root 2024-09-19T09:23:42.404 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ignoring --setgroup ceph since I am not root 2024-09-19T09:23:42.404 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.362+0000 7fbb5b2ca040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:23:42.753 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.423+0000 7fbb5b2ca040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:42.754 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.579+0000 7fbb5b2ca040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:23:42.754 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.649+0000 7fbb5b2ca040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.394+0000 7fef08640640 1 Processor -- start 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.394+0000 7fef08640640 1 -- start start 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.394+0000 7fef08640640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.394+0000 7fef08640640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fef00108200 con 0x7fef00107830 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.395+0000 7fef063dc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.006 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.395+0000 7fef063dc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38140/0 (socket says 172.21.15.28:38140) 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.395+0000 7fef063dc640 1 -- 172.21.15.28:0/3989843905 learned_addr learned my addr 172.21.15.28:0/3989843905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.395+0000 7fef063dc640 1 -- 172.21.15.28:0/3989843905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef001089e0 con 0x7fef00107830 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.396+0000 7fef063dc640 1 --2- 172.21.15.28:0/3989843905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feefc009820 tx=0x7feefc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3478bbf7f316f6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.396+0000 7fef053da640 1 -- 172.21.15.28:0/3989843905 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc037070 con 0x7fef00107830 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.396+0000 7fef053da640 1 -- 172.21.15.28:0/3989843905 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feefc032920 con 0x7fef00107830 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.396+0000 7fef053da640 1 -- 172.21.15.28:0/3989843905 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc030810 con 0x7fef00107830 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.397+0000 7fef08640640 1 -- 172.21.15.28:0/3989843905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 msgr2=0x7fef00107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.397+0000 7fef08640640 1 --2- 172.21.15.28:0/3989843905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feefc009820 tx=0x7feefc02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.397+0000 7fef08640640 1 -- 172.21.15.28:0/3989843905 shutdown_connections 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 --2- 172.21.15.28:0/3989843905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef00107c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 -- 172.21.15.28:0/3989843905 >> 172.21.15.28:0/3989843905 conn(0x7fef00103040 msgr2=0x7fef00105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.007 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 -- 172.21.15.28:0/3989843905 shutdown_connections 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 -- 172.21.15.28:0/3989843905 wait complete. 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 Processor -- start 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.398+0000 7fef08640640 1 -- start start 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef08640640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef08640640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fef0010bd90 con 0x7fef00107830 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef063dc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef063dc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38142/0 (socket says 172.21.15.28:38142) 2024-09-19T09:23:43.008 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef063dc640 1 -- 172.21.15.28:0/2696658328 learned_addr learned my addr 172.21.15.28:0/2696658328 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef063dc640 1 -- 172.21.15.28:0/2696658328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0019bdc0 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.399+0000 7fef063dc640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feefc030560 tx=0x7feefc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.400+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc037040 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.400+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0019c050 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.400+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feefc040070 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.400+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0019ed40 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.400+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc03a740 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7feefc03a8e0 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7feef77fe640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 0x7feeec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0010bf10 con 0x7fef00107830 2024-09-19T09:23:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feefc06d870 con 0x7fef00107830 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7fef05bdb640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 0x7feeec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.401+0000 7fef05bdb640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 0x7feeec03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feef00098c0 tx=0x7feef0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.406+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feefc041270 con 0x7fef00107830 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:41.644+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fef00107cb0 con 0x7fef00107830 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.206+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7feefc045980 con 0x7fef00107830 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.208+0000 7feef77fe640 1 -- 172.21.15.28:0/2696658328 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7feefc045460 con 0x7fef00107830 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 msgr2=0x7feeec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 0x7feeec03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feef00098c0 tx=0x7feef0006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 msgr2=0x7fef0019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feefc030560 tx=0x7feefc032670 comp rx=0 tx=0).stop 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 shutdown_connections 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7feeec039200 0x7feeec03b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.010 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 --2- 172.21.15.28:0/2696658328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef00107830 0x7fef0019b880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 >> 172.21.15.28:0/2696658328 conn(0x7fef00103040 msgr2=0x7fef00103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 shutdown_connections 2024-09-19T09:23:43.011 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:42.210+0000 7fef08640640 1 -- 172.21.15.28:0/2696658328 wait complete. 2024-09-19T09:23:43.013 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.805+0000 7fbb5b2ca040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:43.013 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:42.952+0000 7fbb5b2ca040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:23:43.402 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:43 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2696658328' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T09:23:43.402 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:43 smithi028 ceph-mon[30972]: mgrmap e9: x(active, since 19s) 2024-09-19T09:23:43.402 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:43.291+0000 7fbb5b2ca040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:23:43.402 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:43.356+0000 7fbb5b2ca040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:23:43.653 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:43.558+0000 7fbb5b2ca040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:23:44.289 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:44.289 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.414+0000 7f467f963640 1 Processor -- start 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467f963640 1 -- start start 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467f963640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467f963640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4678107080 con 0x7f46781066b0 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467d6ff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467d6ff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59346/0 (socket says 172.21.15.28:59346) 2024-09-19T09:23:44.290 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467d6ff640 1 -- 172.21.15.28:0/591177634 learned_addr learned my addr 172.21.15.28:0/591177634 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.415+0000 7f467d6ff640 1 -- 172.21.15.28:0/591177634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781078b0 con 0x7f46781066b0 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.416+0000 7f467d6ff640 1 --2- 172.21.15.28:0/591177634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4674009820 tx=0x7f467402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772c73c6321d648 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f466ffff640 1 -- 172.21.15.28:0/591177634 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674037070 con 0x7f46781066b0 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f466ffff640 1 -- 172.21.15.28:0/591177634 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674032920 con 0x7f46781066b0 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f466ffff640 1 -- 172.21.15.28:0/591177634 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674030810 con 0x7f46781066b0 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 -- 172.21.15.28:0/591177634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 msgr2=0x7f4678106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 --2- 172.21.15.28:0/591177634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4674009820 tx=0x7f467402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 -- 172.21.15.28:0/591177634 shutdown_connections 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 --2- 172.21.15.28:0/591177634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f4678106ab0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 -- 172.21.15.28:0/591177634 >> 172.21.15.28:0/591177634 conn(0x7f4678101e60 msgr2=0x7f4678104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 -- 172.21.15.28:0/591177634 shutdown_connections 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.417+0000 7f467f963640 1 -- 172.21.15.28:0/591177634 wait complete. 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.418+0000 7f467f963640 1 Processor -- start 2024-09-19T09:23:44.291 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.418+0000 7f467f963640 1 -- start start 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467f963640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467f963640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f467810ac60 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467d6ff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467d6ff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59354/0 (socket says 172.21.15.28:59354) 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467d6ff640 1 -- 172.21.15.28:0/1642790850 learned_addr learned my addr 172.21.15.28:0/1642790850 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467d6ff640 1 -- 172.21.15.28:0/1642790850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467819e590 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467d6ff640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4674030560 tx=0x7f4674032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674037040 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674040070 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467819e760 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.419+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467403a740 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.420+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467819eb00 con 0x7f46781066b0 2024-09-19T09:23:44.292 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.420+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f467403a9b0 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.421+0000 7f466e7fc640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 0x7f466403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.421+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4678106b30 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.421+0000 7f467cefe640 1 -- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 msgr2=0x7f466403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.421+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f467406c900 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.421+0000 7f467cefe640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 0x7f466403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.425+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4674042e30 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.622+0000 7f467cefe640 1 -- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 msgr2=0x7f466403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.622+0000 7f467cefe640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 0x7f466403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.652+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f467819ef40 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.652+0000 7f466e7fc640 1 -- 172.21.15.28:0/1642790850 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f467819ef40 con 0x7f46781066b0 2024-09-19T09:23:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 msgr2=0x7f466403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 0x7f466403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 msgr2=0x7f467807ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4674030560 tx=0x7f4674032670 comp rx=0 tx=0).stop 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 shutdown_connections 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7f4664039200 0x7f466403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 --2- 172.21.15.28:0/1642790850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46781066b0 0x7f467807ba00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 >> 172.21.15.28:0/1642790850 conn(0x7f4678101e60 msgr2=0x7f4678102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 shutdown_connections 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:43.655+0000 7f467f963640 1 -- 172.21.15.28:0/1642790850 wait complete. 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for the mgr to restart... 2024-09-19T09:23:44.294 INFO:teuthology.orchestra.run.smithi028.stdout:Waiting for mgr epoch 9... 2024-09-19T09:23:44.485 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:44 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1642790850' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T09:23:44.763 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:44.503+0000 7fbb5b2ca040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:23:44.763 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:44.574+0000 7fbb5b2ca040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:23:45.600 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.329+0000 7fbb5b2ca040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:23:45.601 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.449+0000 7fbb5b2ca040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:23:45.892 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.599+0000 7fbb5b2ca040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:23:45.892 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.663+0000 7fbb5b2ca040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:23:45.893 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.819+0000 7fbb5b2ca040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:23:45.893 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.891+0000 7fbb5b2ca040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:23:46.190 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:45.953+0000 7fbb5b2ca040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:23:46.190 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.120+0000 7fbb5b2ca040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:23:46.190 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.188+0000 7fbb5b2ca040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:23:46.470 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.258+0000 7fbb5b2ca040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:23:46.470 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.345+0000 7fbb5b2ca040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:23:46.470 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.407+0000 7fbb5b2ca040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:23:46.470 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:46.468+0000 7fbb5b2ca040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:23:47.556 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:47.300+0000 7fbb5b2ca040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:23:47.556 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: Active manager daemon x restarted 2024-09-19T09:23:47.556 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: Activating manager daemon x 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: mgrmap e10: x(active, starting, since 0.00361326s) 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: Manager daemon x is now available 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:23:47.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:23:49.258 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:48.970+0000 7fbac65de640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:09:23:48] ENGINE Error in HTTPServer.serve 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: Traceback (most recent call last): 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._connections.run(self.expiration_interval) 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._run(expiration_interval) 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:23:49.259 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: s = self.context.wrap_socket( 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: return self.sslsocket_class._create( 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self.do_handshake() 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: self._sslobj.do_handshake() 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:23:49.260 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:23:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc8b3c9640 1 Processor -- start 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc8b3c9640 1 -- start start 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc8b3c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc8b3c9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84107090 con 0x7fcc841066c0 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc89165640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:49.268 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc89165640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59362/0 (socket says 172.21.15.28:59362) 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.688+0000 7fcc89165640 1 -- 172.21.15.28:0/1987569809 learned_addr learned my addr 172.21.15.28:0/1987569809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.689+0000 7fcc89165640 1 -- 172.21.15.28:0/1987569809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc841078c0 con 0x7fcc841066c0 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.689+0000 7fcc89165640 1 --2- 172.21.15.28:0/1987569809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000bac0 tx=0x7fcc8002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbeed9840c7d4ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.689+0000 7fcc7bfff640 1 -- 172.21.15.28:0/1987569809 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc800326d0 con 0x7fcc841066c0 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc7bfff640 1 -- 172.21.15.28:0/1987569809 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc80038070 con 0x7fcc841066c0 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc7bfff640 1 -- 172.21.15.28:0/1987569809 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc80033d00 con 0x7fcc841066c0 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/1987569809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 msgr2=0x7fcc84106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/1987569809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000bac0 tx=0x7fcc8002c920 comp rx=0 tx=0).stop 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/1987569809 shutdown_connections 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/1987569809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc84106ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.690+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/1987569809 >> 172.21.15.28:0/1987569809 conn(0x7fcc84101e90 msgr2=0x7fcc841042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:49.269 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.691+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/1987569809 shutdown_connections 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.691+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/1987569809 wait complete. 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.691+0000 7fcc8b3c9640 1 Processor -- start 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc8b3c9640 1 -- start start 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc8b3c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc8b3c9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8407b100 con 0x7fcc841066c0 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc89165640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc89165640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59366/0 (socket says 172.21.15.28:59366) 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc89165640 1 -- 172.21.15.28:0/2917383243 learned_addr learned my addr 172.21.15.28:0/2917383243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc89165640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc84199be0 con 0x7fcc841066c0 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.692+0000 7fcc89165640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000bbe0 tx=0x7fcc800327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.693+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc80042070 con 0x7fcc841066c0 2024-09-19T09:23:49.270 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.693+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc80032cd0 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.693+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc84199e70 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.693+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc80045660 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.693+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8419cb60 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fcc8004d060 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc7a7fc640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc7003be30 con 0x7fcc70039260 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc80030050 con 0x7fcc841066c0 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc88964640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 msgr2=0x7fcc7003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.694+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.895+0000 7fcc88964640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 msgr2=0x7fcc7003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:44.895+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:45.296+0000 7fcc88964640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 msgr2=0x7fcc7003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:49.271 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:45.296+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:46.097+0000 7fcc88964640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 msgr2=0x7fcc7003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.28:6800/3407590911 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:46.097+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:47.304+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fcc8006d5c0 con 0x7fcc841066c0 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:47.304+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 msgr2=0x7fcc7003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:47.304+0000 7fcc7a7fc640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.306+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fcc8003c3f0 con 0x7fcc841066c0 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.306+0000 7fcc7a7fc640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 0x7fcc7003f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.306+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc80044ec0 con 0x7fcc7003cd20 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.306+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 0x7fcc7003f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.307+0000 7fcc88964640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 0x7fcc7003f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc74003980 tx=0x7fcc740092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:49.272 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.308+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fcc80044ec0 con 0x7fcc7003cd20 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcc84106ac0 con 0x7fcc7003cd20 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc7a7fc640 1 -- 172.21.15.28:0/2917383243 <== mgr.14150 v2:172.21.15.28:6800/147346946 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fcc84106ac0 con 0x7fcc7003cd20 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 msgr2=0x7fcc7003f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 0x7fcc7003f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc74003980 tx=0x7fcc740092b0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 msgr2=0x7fcc841996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000bbe0 tx=0x7fcc800327d0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 shutdown_connections 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcc7003cd20 0x7fcc7003f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:6800/3407590911,v1:172.21.15.28:6801/3407590911] conn(0x7fcc70039260 0x7fcc7003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 --2- 172.21.15.28:0/2917383243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc841066c0 0x7fcc841996a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 >> 172.21.15.28:0/2917383243 conn(0x7fcc84101e90 msgr2=0x7fcc8418fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:49.273 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 shutdown_connections 2024-09-19T09:23:49.274 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:48.315+0000 7fcc8b3c9640 1 -- 172.21.15.28:0/2917383243 wait complete. 2024-09-19T09:23:49.274 INFO:teuthology.orchestra.run.smithi028.stdout:mgr epoch 9 is available 2024-09-19T09:23:49.274 INFO:teuthology.orchestra.run.smithi028.stdout:Generating a dashboard self-signed certificate... 2024-09-19T09:23:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: mgrmap e11: x(active, since 1.00624s) 2024-09-19T09:23:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:23:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:48] ENGINE Bus STARTING 2024-09-19T09:23:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:48] ENGINE Serving on https://172.21.15.28:7150 2024-09-19T09:23:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:48] ENGINE Error in HTTPServer.serve 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: Traceback (most recent call last): 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._connections.run(self.expiration_interval) 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._run(expiration_interval) 2024-09-19T09:23:49.515 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: s = self.context.wrap_socket( 2024-09-19T09:23:49.516 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: return self.sslsocket_class._create( 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: self.do_handshake() 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: self._sslobj.do_handshake() 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:49] ENGINE Serving on http://172.21.15.28:8765 2024-09-19T09:23:49.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:49 smithi028 ceph-mon[30972]: [19/Sep/2024:09:23:49] ENGINE Bus STARTED 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc737df1640 1 Processor -- start 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc737df1640 1 -- start start 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc737df1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc737df1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc73007bd80 con 0x7fc730108300 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc735b8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc735b8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59444/0 (socket says 172.21.15.28:59444) 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.712+0000 7fc735b8d640 1 -- 172.21.15.28:0/428820197 learned_addr learned my addr 172.21.15.28:0/428820197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.713+0000 7fc735b8d640 1 -- 172.21.15.28:0/428820197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73010ac30 con 0x7fc730108300 2024-09-19T09:23:50.472 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.713+0000 7fc735b8d640 1 --2- 172.21.15.28:0/428820197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009820 tx=0x7fc72c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a70a987cb8e7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.713+0000 7fc734b8b640 1 -- 172.21.15.28:0/428820197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc72c037070 con 0x7fc730108300 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc734b8b640 1 -- 172.21.15.28:0/428820197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c032920 con 0x7fc730108300 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc734b8b640 1 -- 172.21.15.28:0/428820197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc72c030810 con 0x7fc730108300 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc737df1640 1 -- 172.21.15.28:0/428820197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 msgr2=0x7fc73010a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc737df1640 1 --2- 172.21.15.28:0/428820197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009820 tx=0x7fc72c02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc737df1640 1 -- 172.21.15.28:0/428820197 shutdown_connections 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc737df1640 1 --2- 172.21.15.28:0/428820197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73010a6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.714+0000 7fc737df1640 1 -- 172.21.15.28:0/428820197 >> 172.21.15.28:0/428820197 conn(0x7fc730101ec0 msgr2=0x7fc7301042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 -- 172.21.15.28:0/428820197 shutdown_connections 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 -- 172.21.15.28:0/428820197 wait complete. 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 Processor -- start 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 -- start start 2024-09-19T09:23:50.473 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.715+0000 7fc737df1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc73010bbf0 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc735b8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc735b8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59456/0 (socket says 172.21.15.28:59456) 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc735b8d640 1 -- 172.21.15.28:0/3305939645 learned_addr learned my addr 172.21.15.28:0/3305939645 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc735b8d640 1 -- 172.21.15.28:0/3305939645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73019bd20 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc735b8d640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc72c030380 tx=0x7fc72c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc72c037040 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c040070 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73019bfb0 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc72c03a7b0 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.716+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73019eca0 con 0x7fc730108300 2024-09-19T09:23:50.474 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.717+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fc72c03aa30 con 0x7fc730108300 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.717+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc730108760 con 0x7fc730108300 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.718+0000 7fc726ffd640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 0x7fc71c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.718+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc72c06ca40 con 0x7fc730108300 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.718+0000 7fc73538c640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 0x7fc71c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.718+0000 7fc73538c640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 0x7fc71c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7200098c0 tx=0x7fc720006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.723+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc72c03ae70 con 0x7fc730108300 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.906+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fc730103170 con 0x7fc71c039200 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.947+0000 7fc726ffd640 1 -- 172.21.15.28:0/3305939645 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fc730103170 con 0x7fc71c039200 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 msgr2=0x7fc71c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 0x7fc71c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7200098c0 tx=0x7fc720006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 msgr2=0x7fc73019b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc72c030380 tx=0x7fc72c032670 comp rx=0 tx=0).stop 2024-09-19T09:23:50.475 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 shutdown_connections 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc71c039200 0x7fc71c03b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 --2- 172.21.15.28:0/3305939645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc730108300 0x7fc73019b7e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 >> 172.21.15.28:0/3305939645 conn(0x7fc730101ec0 msgr2=0x7fc730102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 shutdown_connections 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:49.949+0000 7fc737df1640 1 -- 172.21.15.28:0/3305939645 wait complete. 2024-09-19T09:23:50.476 INFO:teuthology.orchestra.run.smithi028.stdout:Creating initial admin user... 2024-09-19T09:23:51.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:50 smithi028 ceph-mon[30972]: mgrmap e12: x(active, since 2s) 2024-09-19T09:23:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:50 smithi028 ceph-mon[30972]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:23:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:23:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:23:52.009 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:51 smithi028 ceph-mon[30972]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:52.009 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:51 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VLtt1mr7hhZJ75yz/HFnsef8clMxwoZ1ZaDIJqOrt/io80T0G6Bka", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726737831, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.900+0000 7f42c87dd640 1 Processor -- start 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.900+0000 7f42c87dd640 1 -- start start 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.900+0000 7f42c87dd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.900+0000 7f42c87dd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0107080 con 0x7f42c01066b0 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.901+0000 7f42c6579640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.901+0000 7f42c6579640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59460/0 (socket says 172.21.15.28:59460) 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.901+0000 7f42c6579640 1 -- 172.21.15.28:0/3129638941 learned_addr learned my addr 172.21.15.28:0/3129638941 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.901+0000 7f42c6579640 1 -- 172.21.15.28:0/3129638941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01078b0 con 0x7f42c01066b0 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.902+0000 7f42c6579640 1 --2- 172.21.15.28:0/3129638941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009820 tx=0x7f42bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd94ed244269add4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.902+0000 7f42c5577640 1 -- 172.21.15.28:0/3129638941 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42bc037070 con 0x7f42c01066b0 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.902+0000 7f42c5577640 1 -- 172.21.15.28:0/3129638941 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42bc032920 con 0x7f42c01066b0 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.902+0000 7f42c5577640 1 -- 172.21.15.28:0/3129638941 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42bc030810 con 0x7f42c01066b0 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 -- 172.21.15.28:0/3129638941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 msgr2=0x7f42c0106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 --2- 172.21.15.28:0/3129638941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009820 tx=0x7f42bc02a270 comp rx=0 tx=0).stop 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 -- 172.21.15.28:0/3129638941 shutdown_connections 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 --2- 172.21.15.28:0/3129638941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c0106ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 -- 172.21.15.28:0/3129638941 >> 172.21.15.28:0/3129638941 conn(0x7f42c0101e60 msgr2=0x7f42c0104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 -- 172.21.15.28:0/3129638941 shutdown_connections 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 -- 172.21.15.28:0/3129638941 wait complete. 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.903+0000 7f42c87dd640 1 Processor -- start 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c87dd640 1 -- start start 2024-09-19T09:23:52.018 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c87dd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c87dd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42c010ac60 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c6579640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c6579640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59472/0 (socket says 172.21.15.28:59472) 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c6579640 1 -- 172.21.15.28:0/218209298 learned_addr learned my addr 172.21.15.28:0/218209298 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c6579640 1 -- 172.21.15.28:0/218209298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c019bde0 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.904+0000 7f42c6579640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0097f0 tx=0x7f42bc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.905+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42bc037040 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.905+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c019c070 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.905+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42bc030cc0 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.905+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42bc03a640 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.905+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c019ed60 con 0x7f42c01066b0 2024-09-19T09:23:52.019 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f42bc03a8f0 con 0x7f42c01066b0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42b77fe640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 0x7f42ac03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f42bc06c8e0 con 0x7f42c01066b0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42c5d78640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 0x7f42ac03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c010ade0 con 0x7f42c01066b0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.906+0000 7f42c5d78640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 0x7f42ac03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098c0 tx=0x7f42b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:50.911+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f42bc0328c0 con 0x7f42c01066b0 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.105+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f42c0106b30 con 0x7f42ac039260 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.355+0000 7f42b77fe640 1 -- 172.21.15.28:0/218209298 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f42c0106b30 con 0x7f42ac039260 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 msgr2=0x7f42ac03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 0x7f42ac03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098c0 tx=0x7f42b0006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.020 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 msgr2=0x7f42c019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0097f0 tx=0x7f42bc0327b0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 shutdown_connections 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f42ac039260 0x7f42ac03b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 --2- 172.21.15.28:0/218209298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c01066b0 0x7f42c019b8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 >> 172.21.15.28:0/218209298 conn(0x7f42c0101e60 msgr2=0x7f42c0102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 shutdown_connections 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:51.357+0000 7f42c87dd640 1 -- 172.21.15.28:0/218209298 wait complete. 2024-09-19T09:23:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:Fetching dashboard port number... 2024-09-19T09:23:52.998 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:52 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/305102107' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T09:23:53.677 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T09:23:53.677 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62ea499640 1 Processor -- start 2024-09-19T09:23:53.677 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62ea499640 1 -- start start 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62ea499640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62ea499640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4107080 con 0x7f62e41066b0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59488/0 (socket says 172.21.15.28:59488) 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62e3fff640 1 -- 172.21.15.28:0/3983254419 learned_addr learned my addr 172.21.15.28:0/3983254419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.544+0000 7f62e3fff640 1 -- 172.21.15.28:0/3983254419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41078b0 con 0x7f62e41066b0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.545+0000 7f62e3fff640 1 --2- 172.21.15.28:0/3983254419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009820 tx=0x7f62d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f6b9886445e7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.545+0000 7f62e37fe640 1 -- 172.21.15.28:0/3983254419 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d4037070 con 0x7f62e41066b0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.545+0000 7f62e37fe640 1 -- 172.21.15.28:0/3983254419 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d4032920 con 0x7f62e41066b0 2024-09-19T09:23:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.545+0000 7f62e37fe640 1 -- 172.21.15.28:0/3983254419 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d4030810 con 0x7f62e41066b0 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 -- 172.21.15.28:0/3983254419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 msgr2=0x7f62e4106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 --2- 172.21.15.28:0/3983254419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009820 tx=0x7f62d402a270 comp rx=0 tx=0).stop 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 -- 172.21.15.28:0/3983254419 shutdown_connections 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 --2- 172.21.15.28:0/3983254419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e4106ab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 -- 172.21.15.28:0/3983254419 >> 172.21.15.28:0/3983254419 conn(0x7f62e4101ec0 msgr2=0x7f62e41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 -- 172.21.15.28:0/3983254419 shutdown_connections 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.546+0000 7f62ea499640 1 -- 172.21.15.28:0/3983254419 wait complete. 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62ea499640 1 Processor -- start 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62ea499640 1 -- start start 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62ea499640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62ea499640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410ac60 con 0x7f62e41066b0 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:53.679 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59496/0 (socket says 172.21.15.28:59496) 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.547+0000 7f62e3fff640 1 -- 172.21.15.28:0/305102107 learned_addr learned my addr 172.21.15.28:0/305102107 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62e3fff640 1 -- 172.21.15.28:0/305102107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e419bdc0 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62e3fff640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097f0 tx=0x7f62d40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d4037040 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e419c050 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d4030cc0 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d403a640 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.548+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e419ed40 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.549+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e410ade0 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.549+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f62d403b3f0 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.549+0000 7f62e1ffb640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 0x7f62d003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.549+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62d406c920 con 0x7f62e41066b0 2024-09-19T09:23:53.680 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.550+0000 7f62dbfff640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 0x7f62d003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.550+0000 7f62dbfff640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 0x7f62d003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098c0 tx=0x7f62cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.554+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62d4042e10 con 0x7f62e41066b0 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.735+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f62e4106b30 con 0x7f62e41066b0 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.736+0000 7f62e1ffb640 1 -- 172.21.15.28:0/305102107 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f62d4043de0 con 0x7f62e41066b0 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 msgr2=0x7f62d003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 0x7f62d003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098c0 tx=0x7f62cc006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 msgr2=0x7f62e419b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097f0 tx=0x7f62d40327b0 comp rx=0 tx=0).stop 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 shutdown_connections 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62d0039200 0x7f62d003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 --2- 172.21.15.28:0/305102107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e41066b0 0x7f62e419b880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:53.681 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 >> 172.21.15.28:0/305102107 conn(0x7f62e4101ec0 msgr2=0x7f62e4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:53.682 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 shutdown_connections 2024-09-19T09:23:53.682 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:52.738+0000 7f62ea499640 1 -- 172.21.15.28:0/305102107 wait complete. 2024-09-19T09:23:53.686 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:23:53.686 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout disabled 2024-09-19T09:23:53.694 INFO:teuthology.orchestra.run.smithi028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:23:53.694 INFO:teuthology.orchestra.run.smithi028.stdout:systemctl: stdout inactive 2024-09-19T09:23:53.694 INFO:teuthology.orchestra.run.smithi028.stdout:firewalld.service is not enabled 2024-09-19T09:23:53.694 INFO:teuthology.orchestra.run.smithi028.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout:Ceph Dashboard is now available at: 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout: URL: https://smithi028.front.sepia.ceph.com:8443/ 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout: User: admin 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout: Password: t77yvz02cr 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:53.696 INFO:teuthology.orchestra.run.smithi028.stdout:Saving cluster configuration to /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config directory 2024-09-19T09:23:54.992 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:54 smithi028 ceph-mon[30972]: mgrmap e13: x(active, since 6s) 2024-09-19T09:23:54.992 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:54 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/172218243' entity='client.admin' 2024-09-19T09:23:54.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.109+0000 7f066cc57640 1 Processor -- start 2024-09-19T09:23:54.999 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.109+0000 7f066cc57640 1 -- start start 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066cc57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066cc57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f06681070d0 con 0x7f0668106700 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33896/0 (socket says 172.21.15.28:33896) 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066659c640 1 -- 172.21.15.28:0/478165317 learned_addr learned my addr 172.21.15.28:0/478165317 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.110+0000 7f066659c640 1 -- 172.21.15.28:0/478165317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668107900 con 0x7f0668106700 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066659c640 1 --2- 172.21.15.28:0/478165317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0660009e80 tx=0x7f066002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4082f10fa76b7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066559a640 1 -- 172.21.15.28:0/478165317 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660036070 con 0x7f0668106700 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066559a640 1 -- 172.21.15.28:0/478165317 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06600318e0 con 0x7f0668106700 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066559a640 1 -- 172.21.15.28:0/478165317 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660007a00 con 0x7f0668106700 2024-09-19T09:23:55.000 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066cc57640 1 -- 172.21.15.28:0/478165317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 msgr2=0x7f0668106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.111+0000 7f066cc57640 1 --2- 172.21.15.28:0/478165317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0660009e80 tx=0x7f066002a7a0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 -- 172.21.15.28:0/478165317 shutdown_connections 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 --2- 172.21.15.28:0/478165317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668106b00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 -- 172.21.15.28:0/478165317 >> 172.21.15.28:0/478165317 conn(0x7f0668101f30 msgr2=0x7f0668104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 -- 172.21.15.28:0/478165317 shutdown_connections 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 -- 172.21.15.28:0/478165317 wait complete. 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.112+0000 7f066cc57640 1 Processor -- start 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066cc57640 1 -- start start 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066cc57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066cc57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f066810acb0 con 0x7f0668106700 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:55.001 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33904/0 (socket says 172.21.15.28:33904) 2024-09-19T09:23:55.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066659c640 1 -- 172.21.15.28:0/172218243 learned_addr learned my addr 172.21.15.28:0/172218243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:55.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066659c640 1 -- 172.21.15.28:0/172218243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668191e70 con 0x7f0668106700 2024-09-19T09:23:55.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.113+0000 7f066659c640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f066002acc0 tx=0x7f0660031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:55.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.114+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660036050 con 0x7f0668106700 2024-09-19T09:23:55.002 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.114+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660037040 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.114+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0668190060 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.114+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f066003b920 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.114+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681905a0 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0660044020 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066810ae30 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f065f7fe640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 0x7f065403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0660042020 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f0665d9b640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 0x7f065403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.115+0000 7f0665d9b640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 0x7f065403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06500098c0 tx=0x7f0650006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.120+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f066004b020 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.392+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f0668106b80 con 0x7f0668106700 2024-09-19T09:23:55.003 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.395+0000 7f065f7fe640 1 -- 172.21.15.28:0/172218243 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f0660039bc0 con 0x7f0668106700 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 msgr2=0x7f065403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 0x7f065403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f06500098c0 tx=0x7f0650006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 msgr2=0x7f0668191930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f066002acc0 tx=0x7f0660031cc0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 shutdown_connections 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f06540392a0 0x7f065403b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 --2- 172.21.15.28:0/172218243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0668106700 0x7f0668191930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 >> 172.21.15.28:0/172218243 conn(0x7f0668101f30 msgr2=0x7f0668102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 shutdown_connections 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:/usr/bin/ceph: stderr 2024-09-19T09:23:54.397+0000 7f066cc57640 1 -- 172.21.15.28:0/172218243 wait complete. 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T09:23:55.004 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: ceph telemetry on 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout:For more information see: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.005 INFO:teuthology.orchestra.run.smithi028.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T09:23:55.006 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:23:55.006 INFO:teuthology.orchestra.run.smithi028.stdout:Bootstrap complete. 2024-09-19T09:23:55.039 INFO:tasks.cephadm:Fetching config... 2024-09-19T09:23:55.039 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:23:55.039 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T09:23:55.057 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T09:23:55.057 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:23:55.057 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T09:23:55.113 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T09:23:55.113 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:23:55.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T09:23:55.179 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T09:23:55.179 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:23:55.179 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T09:23:55.235 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T09:23:55.235 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:23:55.331 INFO:teuthology.orchestra.run.smithi028.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:55.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:23:55.386 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:55.400 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:23:55.443 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbTv2UAkzHI3T7Ep672bHx6tmwPgNAwLsdmZre7UDGAaS/PQImnw33R4mpoV0w/oaArXEBw7t7yZ9uE2oYGylt3dySAz9O12ruA4A1pMK6a1SiuPAEEz6oPWNF9FlBFqZFPoK5TuI2TvyMgf7I/IrofCfZMqvGap7/u6mmunG1bsSe4tAbHPUgl7GeuHjNb3q+qDlH9jwtyvsysanYbdTjK35INsU7iEuxi0zzL3e20uuf4wDZ3uuj8mNuoHltj6NX3a376SaAj4BQ+ph9V4sLcBiO2FdIfPKORJfjmjtwtWC3FN3bEy3LGsBOgPb+nxNDqW2dnMttSa9biS7F6/Tp8dPnYpZZi+nMKh6jNVpcF/cTtJcgat+qi6dBpHbWKRqxcAkim9ts5NmjVwAG/hrryXLdOQQEp3zzCb/1Cvg1laPWuurG9+IXYhMaTMDaaeoh5wR9WCpO/Yexeff7n0lBMah8Xbct2HyPH1Vi26AYEFCEL+KUSdqSyvT4kZ2z9tc= ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:23:55.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T09:23:55.717 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:56.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.205+0000 7f75e5c9d640 1 -- 172.21.15.28:0/407799425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 msgr2=0x7f75e0104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.205+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/407799425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0104520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009880 tx=0x7f75d002a5d0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- 172.21.15.28:0/407799425 shutdown_connections 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/407799425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0104520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- 172.21.15.28:0/407799425 >> 172.21.15.28:0/407799425 conn(0x7f75e00fbcd0 msgr2=0x7f75e00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- 172.21.15.28:0/407799425 shutdown_connections 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- 172.21.15.28:0/407799425 wait complete. 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 Processor -- start 2024-09-19T09:23:56.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- start start 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75e5c9d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01076e0 con 0x7f75e0102130 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.206+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33930/0 (socket says 172.21.15.28:33930) 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75df7fe640 1 -- 172.21.15.28:0/2973067321 learned_addr learned my addr 172.21.15.28:0/2973067321 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75df7fe640 1 -- 172.21.15.28:0/2973067321 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e01972e0 con 0x7f75e0102130 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75df7fe640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75d0004f60 tx=0x7f75d00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d003f070 con 0x7f75e0102130 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d0003ad0 con 0x7f75e0102130 2024-09-19T09:23:56.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d0035030 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e0197570 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.207+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e019a220 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.208+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f75d0042070 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.208+0000 7f75dd7fa640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 0x7f75cc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.208+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f75d006d5e0 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.208+0000 7f75d7fff640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 0x7f75cc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.209+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e0107e60 con 0x7f75e0102130 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.209+0000 7f75d7fff640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 0x7f75cc03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75c80098c0 tx=0x7f75c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.214+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75d0042360 con 0x7f75e0102130 2024-09-19T09:23:56.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.414+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f75e002d7c0 con 0x7f75e0102130 2024-09-19T09:23:56.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.416+0000 7f75dd7fa640 1 -- 172.21.15.28:0/2973067321 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f75d0003cf0 con 0x7f75e0102130 2024-09-19T09:23:56.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.419+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 msgr2=0x7f75cc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.419+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 0x7f75cc03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75c80098c0 tx=0x7f75c8006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.419+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 msgr2=0x7f75e0196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.419+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75d0004f60 tx=0x7f75d00048f0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 shutdown_connections 2024-09-19T09:23:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75cc039300 0x7f75cc03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 --2- 172.21.15.28:0/2973067321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75e0102130 0x7f75e0196da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 >> 172.21.15.28:0/2973067321 conn(0x7f75e00fbcd0 msgr2=0x7f75e00fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 shutdown_connections 2024-09-19T09:23:56.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:56.420+0000 7f75e5c9d640 1 -- 172.21.15.28:0/2973067321 wait complete. 2024-09-19T09:23:57.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:57 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2973067321' entity='client.admin' 2024-09-19T09:23:58.158 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T09:23:58.159 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T09:23:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:59.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 -- 172.21.15.28:0/3604610766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 msgr2=0x7f33c8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 --2- 172.21.15.28:0/3604610766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c8100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009880 tx=0x7f33b802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 -- 172.21.15.28:0/3604610766 shutdown_connections 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 --2- 172.21.15.28:0/3604610766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c8100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 -- 172.21.15.28:0/3604610766 >> 172.21.15.28:0/3604610766 conn(0x7f33c80fbc80 msgr2=0x7f33c80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 -- 172.21.15.28:0/3604610766 shutdown_connections 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 -- 172.21.15.28:0/3604610766 wait complete. 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.270+0000 7f33cdb69640 1 Processor -- start 2024-09-19T09:23:59.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33cdb69640 1 -- start start 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33cdb69640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33cdb69640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8106620 con 0x7f33c8100510 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33950/0 (socket says 172.21.15.28:33950) 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33c77fe640 1 -- 172.21.15.28:0/2022668186 learned_addr learned my addr 172.21.15.28:0/2022668186 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:23:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33c77fe640 1 -- 172.21.15.28:0/2022668186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c8199b10 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.271+0000 7f33c77fe640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b8004f60 tx=0x7f33b80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.272+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b803f070 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.272+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c8199ce0 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.272+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b8004d40 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.272+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b8035030 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.272+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c819a080 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f33b8042070 con 0x7f33c8100510 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33c57fa640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 0x7f33b403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:59.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f33b806cfb0 con 0x7f33c8100510 2024-09-19T09:23:59.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8106da0 con 0x7f33c8100510 2024-09-19T09:23:59.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33beffd640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 0x7f33b403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:59.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.273+0000 7f33beffd640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 0x7f33b403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098c0 tx=0x7f33b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:59.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.278+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33b8040260 con 0x7f33c8100510 2024-09-19T09:23:59.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.456+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f33c802d900 con 0x7f33b40392a0 2024-09-19T09:23:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.460+0000 7f33c57fa640 1 -- 172.21.15.28:0/2022668186 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33c802d900 con 0x7f33b40392a0 2024-09-19T09:23:59.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 msgr2=0x7f33b403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:59.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 0x7f33b403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33b00098c0 tx=0x7f33b0006db0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 msgr2=0x7f33c806a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:59.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b8004f60 tx=0x7f33b80048f0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 shutdown_connections 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33b40392a0 0x7f33b403b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 --2- 172.21.15.28:0/2022668186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33c8100510 0x7f33c806a950 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 >> 172.21.15.28:0/2022668186 conn(0x7f33c80fbc80 msgr2=0x7f33c80fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 shutdown_connections 2024-09-19T09:23:59.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:23:59.462+0000 7f33cdb69640 1 -- 172.21.15.28:0/2022668186 wait complete. 2024-09-19T09:24:00.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi028", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:00.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:23:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:00.280 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi038 2024-09-19T09:24:00.280 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:24:00.280 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.conf 2024-09-19T09:24:00.299 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:24:00.300 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:00.358 INFO:tasks.cephadm:Adding host smithi038 to orchestrator... 2024-09-19T09:24:00.358 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch host add smithi038 2024-09-19T09:24:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:01.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:01.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: Updating smithi028:/etc/ceph/ceph.conf 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: Updating smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: Updating smithi028:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: Updating smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:01.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 -- 172.21.15.28:0/3473369990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 msgr2=0x7f6a10102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:01.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 --2- 172.21.15.28:0/3473369990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a10102b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009c20 tx=0x7f6a0002ab20 comp rx=0 tx=0).stop 2024-09-19T09:24:01.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 -- 172.21.15.28:0/3473369990 shutdown_connections 2024-09-19T09:24:01.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 --2- 172.21.15.28:0/3473369990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a10102b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 -- 172.21.15.28:0/3473369990 >> 172.21.15.28:0/3473369990 conn(0x7f6a100fc1b0 msgr2=0x7f6a100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 -- 172.21.15.28:0/3473369990 shutdown_connections 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.051+0000 7f6a1581c640 1 -- 172.21.15.28:0/3473369990 wait complete. 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a1581c640 1 Processor -- start 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a1581c640 1 -- start start 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a1581c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a1581c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1010bcc0 con 0x7f6a10102750 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a0effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a0effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33978/0 (socket says 172.21.15.28:33978) 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.052+0000 7f6a0effd640 1 -- 172.21.15.28:0/278514397 learned_addr learned my addr 172.21.15.28:0/278514397 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a0effd640 1 -- 172.21.15.28:0/278514397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a101022a0 con 0x7f6a10102750 2024-09-19T09:24:01.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a0effd640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a00030040 tx=0x7f6a0002eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a00047070 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a101008f0 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a000368e0 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a00042420 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.053+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a10100db0 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.054+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6a0004a070 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.054+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1006a7b0 con 0x7f6a10102750 2024-09-19T09:24:01.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.055+0000 7f6a1481a640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 0x7f69f803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:01.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.055+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a0003d030 con 0x7f6a10102750 2024-09-19T09:24:01.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.055+0000 7f6a0e7fc640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 0x7f69f803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:01.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.055+0000 7f6a0e7fc640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 0x7f69f803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a040098c0 tx=0x7f6a04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:01.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.059+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a00032030 con 0x7f6a10102750 2024-09-19T09:24:01.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:01.238+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}) -- 0x7f6a101018e0 con 0x7f69f80392a0 2024-09-19T09:24:02.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:01 smithi028 ceph-mon[30972]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:03.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:03 smithi028 ceph-mon[30972]: Deploying cephadm binary to smithi038 2024-09-19T09:24:05.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.150+0000 7f6a1481a640 1 -- 172.21.15.28:0/278514397 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6a101018e0 con 0x7f69f80392a0 2024-09-19T09:24:05.151 INFO:teuthology.orchestra.run.smithi028.stdout:Added host 'smithi038' with addr '172.21.15.38' 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 msgr2=0x7f69f803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 0x7f69f803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a040098c0 tx=0x7f6a04006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 msgr2=0x7f6a101003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a00030040 tx=0x7f6a0002eb50 comp rx=0 tx=0).stop 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 shutdown_connections 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f69f80392a0 0x7f69f803b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 --2- 172.21.15.28:0/278514397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a10102750 0x7f6a101003b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 >> 172.21.15.28:0/278514397 conn(0x7f6a100fc1b0 msgr2=0x7f6a10105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 shutdown_connections 2024-09-19T09:24:05.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:05.152+0000 7f6a1581c640 1 -- 172.21.15.28:0/278514397 wait complete. 2024-09-19T09:24:05.965 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T09:24:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:06.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:06 smithi028 ceph-mon[30972]: Added host smithi038 2024-09-19T09:24:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:06.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 -- 172.21.15.28:0/1496802785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 msgr2=0x7f7278102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 --2- 172.21.15.28:0/1496802785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f7278102bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7274009810 tx=0x7f727402a540 comp rx=0 tx=0).stop 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 -- 172.21.15.28:0/1496802785 shutdown_connections 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 --2- 172.21.15.28:0/1496802785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f7278102bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 -- 172.21.15.28:0/1496802785 >> 172.21.15.28:0/1496802785 conn(0x7f7278077310 msgr2=0x7f72780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 -- 172.21.15.28:0/1496802785 shutdown_connections 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 -- 172.21.15.28:0/1496802785 wait complete. 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.685+0000 7f72800c5640 1 Processor -- start 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f72800c5640 1 -- start start 2024-09-19T09:24:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f72800c5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f72800c5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f727810bc80 con 0x7f72781027f0 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f727de61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f727de61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53594/0 (socket says 172.21.15.28:53594) 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f727de61640 1 -- 172.21.15.28:0/715349981 learned_addr learned my addr 172.21.15.28:0/715349981 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f727de61640 1 -- 172.21.15.28:0/715349981 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72781998f0 con 0x7f72781027f0 2024-09-19T09:24:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f727de61640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72740045e0 tx=0x7f7274004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:06.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.686+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7274030e70 con 0x7f72781027f0 2024-09-19T09:24:06.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.687+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72740318e0 con 0x7f72781027f0 2024-09-19T09:24:06.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.687+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7278199b80 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.687+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72740049c0 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.687+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7278199fe0 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.687+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7274039040 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.688+0000 7f726effd640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 0x7f726403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.688+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7278069ed0 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.688+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f727406d580 con 0x7f72781027f0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.688+0000 7f727d660640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 0x7f726403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.688+0000 7f727d660640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 0x7f726403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72680098c0 tx=0x7f7268006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:06.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.693+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7274040030 con 0x7f72781027f0 2024-09-19T09:24:06.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.877+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f727819a710 con 0x7f72640392a0 2024-09-19T09:24:06.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.878+0000 7f726effd640 1 -- 172.21.15.28:0/715349981 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f727819a710 con 0x7f72640392a0 2024-09-19T09:24:06.878 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:24:06.878 INFO:teuthology.orchestra.run.smithi028.stdout:[{"addr": "172.21.15.28", "hostname": "smithi028", "labels": [], "status": ""}, {"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}] 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 msgr2=0x7f726403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 0x7f726403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72680098c0 tx=0x7f7268006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 msgr2=0x7f72781993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72740045e0 tx=0x7f7274004610 comp rx=0 tx=0).stop 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 shutdown_connections 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f72640392a0 0x7f726403b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 --2- 172.21.15.28:0/715349981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72781027f0 0x7f72781993b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 >> 172.21.15.28:0/715349981 conn(0x7f7278077310 msgr2=0x7f7278107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 shutdown_connections 2024-09-19T09:24:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:06.880+0000 7f72800c5640 1 -- 172.21.15.28:0/715349981 wait complete. 2024-09-19T09:24:07.503 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2024-09-19T09:24:07.503 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:24:07.503 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2024-09-19T09:24:07.521 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:24:07.521 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:07.578 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2024-09-19T09:24:07.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch host add smithi078 2024-09-19T09:24:07.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:07.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:07 smithi028 ceph-mon[30972]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:24:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 -- 172.21.15.28:0/1526651916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 msgr2=0x7fcd68107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/1526651916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd68107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009880 tx=0x7fcd5802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 -- 172.21.15.28:0/1526651916 shutdown_connections 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/1526651916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd68107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 -- 172.21.15.28:0/1526651916 >> 172.21.15.28:0/1526651916 conn(0x7fcd680fd290 msgr2=0x7fcd680ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 -- 172.21.15.28:0/1526651916 shutdown_connections 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.241+0000 7fcd6fea9640 1 -- 172.21.15.28:0/1526651916 wait complete. 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6fea9640 1 Processor -- start 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6fea9640 1 -- start start 2024-09-19T09:24:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6fea9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6fea9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6810bc90 con 0x7fcd68107660 2024-09-19T09:24:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6dc45640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6dc45640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53612/0 (socket says 172.21.15.28:53612) 2024-09-19T09:24:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6dc45640 1 -- 172.21.15.28:0/3392181383 learned_addr learned my addr 172.21.15.28:0/3392181383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6dc45640 1 -- 172.21.15.28:0/3392181383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68197600 con 0x7fcd68107660 2024-09-19T09:24:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.242+0000 7fcd6dc45640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009230 tx=0x7fcd58003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.243+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5803f070 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.243+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5803bc80 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.243+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd68197890 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.243+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd58035030 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.243+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6819b990 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcd58042070 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68102cd0 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd5effd640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 0x7fcd5403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd5806cfe0 con 0x7fcd68107660 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd6d444640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 0x7fcd5403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.244+0000 7fcd6d444640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 0x7fcd5403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd640098c0 tx=0x7fcd64006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.249+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd580372d0 con 0x7fcd68107660 2024-09-19T09:24:08.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:08.430+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}) -- 0x7fcd68107ac0 con 0x7fcd54039300 2024-09-19T09:24:09.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:08 smithi028 ceph-mon[30972]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:10.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:09 smithi028 ceph-mon[30972]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:11.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: Deploying cephadm binary to smithi078 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: Updating smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: Updating smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:24:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.349+0000 7fcd5effd640 1 -- 172.21.15.28:0/3392181383 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fcd68107ac0 con 0x7fcd54039300 2024-09-19T09:24:12.352 INFO:teuthology.orchestra.run.smithi028.stdout:Added host 'smithi078' with addr '172.21.15.78' 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 msgr2=0x7fcd5403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 0x7fcd5403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd640098c0 tx=0x7fcd64006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 msgr2=0x7fcd681970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009230 tx=0x7fcd58003d40 comp rx=0 tx=0).stop 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 shutdown_connections 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd54039300 0x7fcd5403b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 --2- 172.21.15.28:0/3392181383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd68107660 0x7fcd681970c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 >> 172.21.15.28:0/3392181383 conn(0x7fcd680fd290 msgr2=0x7fcd680febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 shutdown_connections 2024-09-19T09:24:12.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:12.352+0000 7fcd6fea9640 1 -- 172.21.15.28:0/3392181383 wait complete. 2024-09-19T09:24:12.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:12 smithi028 ceph-mon[30972]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:12.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:12.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:12.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T09:24:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 -- 172.21.15.28:0/3120685632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 msgr2=0x7fb750107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 --2- 172.21.15.28:0/3120685632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb750107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb744009880 tx=0x7fb74402a5d0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 -- 172.21.15.28:0/3120685632 shutdown_connections 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 --2- 172.21.15.28:0/3120685632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb750107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 -- 172.21.15.28:0/3120685632 >> 172.21.15.28:0/3120685632 conn(0x7fb7500fd290 msgr2=0x7fb7500ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 -- 172.21.15.28:0/3120685632 shutdown_connections 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.696+0000 7fb756c8e640 1 -- 172.21.15.28:0/3120685632 wait complete. 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb756c8e640 1 Processor -- start 2024-09-19T09:24:13.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb756c8e640 1 -- start start 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb756c8e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb756c8e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010bc90 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb754a2a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb754a2a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41726/0 (socket says 172.21.15.28:41726) 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb754a2a640 1 -- 172.21.15.28:0/4135109163 learned_addr learned my addr 172.21.15.28:0/4135109163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.697+0000 7fb754a2a640 1 -- 172.21.15.28:0/4135109163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7500763e0 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb754a2a640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb744009230 tx=0x7fb744003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74403f070 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb750074a30 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74403bc80 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb744035030 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.698+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb750074ef0 con 0x7fb750107660 2024-09-19T09:24:13.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.699+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb744042070 con 0x7fb750107660 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.699+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb750102cd0 con 0x7fb750107660 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.699+0000 7fb74dffb640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 0x7fb74003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.700+0000 7fb74ffff640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 0x7fb74003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.700+0000 7fb74ffff640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 0x7fb74003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0098c0 tx=0x7fb73c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.701+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb74404ad50 con 0x7fb750107660 2024-09-19T09:24:13.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.706+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb744042360 con 0x7fb750107660 2024-09-19T09:24:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.889+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb750075a20 con 0x7fb7400392a0 2024-09-19T09:24:13.891 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:13 smithi028 ceph-mon[30972]: Added host smithi078 2024-09-19T09:24:13.891 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:13.891 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.890+0000 7fb74dffb640 1 -- 172.21.15.28:0/4135109163 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7fb750075a20 con 0x7fb7400392a0 2024-09-19T09:24:13.892 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:24:13.892 INFO:teuthology.orchestra.run.smithi028.stdout:[{"addr": "172.21.15.28", "hostname": "smithi028", "labels": [], "status": ""}, {"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2024-09-19T09:24:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.892+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 msgr2=0x7fb74003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.892+0000 7fb756c8e640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 0x7fb74003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0098c0 tx=0x7fb73c006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.892+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 msgr2=0x7fb7500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.892+0000 7fb756c8e640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb744009230 tx=0x7fb744003d40 comp rx=0 tx=0).stop 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 shutdown_connections 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7400392a0 0x7fb74003b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 --2- 172.21.15.28:0/4135109163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb750107660 0x7fb7500744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 >> 172.21.15.28:0/4135109163 conn(0x7fb7500fd290 msgr2=0x7fb7500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 shutdown_connections 2024-09-19T09:24:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:13.893+0000 7fb756c8e640 1 -- 172.21.15.28:0/4135109163 wait complete. 2024-09-19T09:24:14.474 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T09:24:14.475 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T09:24:14.726 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:14.955 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:14 smithi028 ceph-mon[30972]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:14.955 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:14 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:14.955 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:14 smithi028 ceph-mon[30972]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3862881425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 msgr2=0x7fcd38106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3862881425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd38106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009880 tx=0x7fcd2802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3862881425 shutdown_connections 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3862881425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd38106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3862881425 >> 172.21.15.28:0/3862881425 conn(0x7fcd380fc1b0 msgr2=0x7fcd380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3862881425 shutdown_connections 2024-09-19T09:24:15.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.179+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3862881425 wait complete. 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd3ea41640 1 Processor -- start 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd3ea41640 1 -- start start 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd3ea41640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd3ea41640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3810ab90 con 0x7fcd38106560 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41740/0 (socket says 172.21.15.28:41740) 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.180+0000 7fcd37fff640 1 -- 172.21.15.28:0/3636745316 learned_addr learned my addr 172.21.15.28:0/3636745316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd37fff640 1 -- 172.21.15.28:0/3636745316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd380fe550 con 0x7fcd38106560 2024-09-19T09:24:15.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd37fff640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009230 tx=0x7fcd28003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:15.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd2803f070 con 0x7fcd38106560 2024-09-19T09:24:15.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd380fcba0 con 0x7fcd38106560 2024-09-19T09:24:15.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2803bc80 con 0x7fcd38106560 2024-09-19T09:24:15.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd28035030 con 0x7fcd38106560 2024-09-19T09:24:15.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.181+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd380fd060 con 0x7fcd38106560 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.182+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcd28042070 con 0x7fcd38106560 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.182+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd38101bd0 con 0x7fcd38106560 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.182+0000 7fcd357fa640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 0x7fcd2003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.182+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd2806cfa0 con 0x7fcd38106560 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.182+0000 7fcd377fe640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 0x7fcd2003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:15.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.183+0000 7fcd377fe640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 0x7fcd2003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0098c0 tx=0x7fcd2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:15.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.187+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd2804c350 con 0x7fcd38106560 2024-09-19T09:24:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.368+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcd380fdb90 con 0x7fcd38106560 2024-09-19T09:24:15.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.702+0000 7fcd357fa640 1 -- 172.21.15.28:0/3636745316 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcd28031e20 con 0x7fcd38106560 2024-09-19T09:24:15.704 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.704+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 msgr2=0x7fcd2003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.704+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 0x7fcd2003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0098c0 tx=0x7fcd2c006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.704+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 msgr2=0x7fcd380fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.704+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009230 tx=0x7fcd28003d40 comp rx=0 tx=0).stop 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 shutdown_connections 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd200392a0 0x7fcd2003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:15.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 --2- 172.21.15.28:0/3636745316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd38106560 0x7fcd380fc660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:15.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 >> 172.21.15.28:0/3636745316 conn(0x7fcd380fc1b0 msgr2=0x7fcd38069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:15.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 shutdown_connections 2024-09-19T09:24:15.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:15.705+0000 7fcd3ea41640 1 -- 172.21.15.28:0/3636745316 wait complete. 2024-09-19T09:24:16.008 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:15 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3636745316' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T09:24:16.319 INFO:tasks.cephadm:Adding mon.a on smithi028 2024-09-19T09:24:16.319 INFO:tasks.cephadm:Adding mon.b on smithi038 2024-09-19T09:24:16.319 INFO:tasks.cephadm:Adding mon.c on smithi078 2024-09-19T09:24:16.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c' 2024-09-19T09:24:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T09:24:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T09:24:17.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3636745316' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 -- 172.21.15.78:0/4204318043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 msgr2=0x7f894c100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 --2- 172.21.15.78:0/4204318043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c100940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f893c009880 tx=0x7f893c02a5d0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 -- 172.21.15.78:0/4204318043 shutdown_connections 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 --2- 172.21.15.78:0/4204318043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c100940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 -- 172.21.15.78:0/4204318043 >> 172.21.15.78:0/4204318043 conn(0x7f894c0fbcd0 msgr2=0x7f894c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 -- 172.21.15.78:0/4204318043 shutdown_connections 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.111+0000 7f895093c640 1 -- 172.21.15.78:0/4204318043 wait complete. 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f895093c640 1 Processor -- start 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f895093c640 1 -- start start 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f895093c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f895093c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1065f0 con 0x7f894c100540 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f894a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f894a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.78:33180/0 (socket says 172.21.15.78:33180) 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.112+0000 7f894a59c640 1 -- 172.21.15.78:0/910839510 learned_addr learned my addr 172.21.15.78:0/910839510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:24:17.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f894a59c640 1 -- 172.21.15.78:0/910839510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c1972d0 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f894a59c640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f893c004f60 tx=0x7f893c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f893c03f070 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f893c004d40 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c197560 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f893c035030 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.113+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c19a210 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.114+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f893c042070 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.114+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c106d70 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.114+0000 7f89437fe640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 0x7f893803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.114+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f893c06e110 con 0x7f894c100540 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.115+0000 7f8943fff640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 0x7f893803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.115+0000 7f8943fff640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 0x7f893803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89340098c0 tx=0x7f8934006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:17.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.120+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f893c042360 con 0x7f894c100540 2024-09-19T09:24:17.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.308+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c", "target": ["mon-mgr", ""]}) -- 0x7f894c02d950 con 0x7f8938039300 2024-09-19T09:24:17.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.313+0000 7f89437fe640 1 -- 172.21.15.78:0/910839510 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f894c02d950 con 0x7f8938039300 2024-09-19T09:24:17.314 INFO:teuthology.orchestra.run.smithi078.stdout:Scheduled mon update... 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 msgr2=0x7f893803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 0x7f893803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89340098c0 tx=0x7f8934006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 msgr2=0x7f894c196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f893c004f60 tx=0x7f893c0048f0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 shutdown_connections 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8938039300 0x7f893803b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 --2- 172.21.15.78:0/910839510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f894c100540 0x7f894c196d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 >> 172.21.15.78:0/910839510 conn(0x7f894c0fbcd0 msgr2=0x7f894c069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 shutdown_connections 2024-09-19T09:24:17.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:17.316+0000 7f895093c640 1 -- 172.21.15.78:0/910839510 wait complete. 2024-09-19T09:24:17.925 DEBUG:teuthology.orchestra.run.smithi038:mon.b> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T09:24:17.928 DEBUG:teuthology.orchestra.run.smithi078:mon.c> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T09:24:17.931 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:24:17.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:24:18.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:18.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:17 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:19.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:18 smithi028 ceph-mon[30972]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:18 smithi028 ceph-mon[30972]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:18 smithi028 ceph-mon[30972]: Saving service mon spec with placement smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c;count:3 2024-09-19T09:24:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:18 smithi028 ceph-mon[30972]: Deploying daemon mon.c on smithi078 2024-09-19T09:24:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:20.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 -- 172.21.15.78:0/1501810430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 msgr2=0x7fd588100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:20.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 --2- 172.21.15.78:0/1501810430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd588100900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009810 tx=0x7fd57c02a540 comp rx=0 tx=0).stop 2024-09-19T09:24:20.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 -- 172.21.15.78:0/1501810430 shutdown_connections 2024-09-19T09:24:20.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 --2- 172.21.15.78:0/1501810430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd588100900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 -- 172.21.15.78:0/1501810430 >> 172.21.15.78:0/1501810430 conn(0x7fd588077310 msgr2=0x7fd5880fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 -- 172.21.15.78:0/1501810430 shutdown_connections 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.459+0000 7fd58f048640 1 -- 172.21.15.78:0/1501810430 wait complete. 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58f048640 1 Processor -- start 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58f048640 1 -- start start 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58f048640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58f048640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd588112280 con 0x7fd588100520 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58cde4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58cde4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.78:33200/0 (socket says 172.21.15.78:33200) 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.460+0000 7fd58cde4640 1 -- 172.21.15.78:0/1077889831 learned_addr learned my addr 172.21.15.78:0/1077889831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.461+0000 7fd58cde4640 1 -- 172.21.15.78:0/1077889831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881a3960 con 0x7fd588100520 2024-09-19T09:24:20.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.461+0000 7fd58cde4640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0045e0 tx=0x7fd57c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.461+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c030e70 con 0x7fd588100520 2024-09-19T09:24:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.461+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c0318e0 con 0x7fd588100520 2024-09-19T09:24:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.461+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881a3bf0 con 0x7fd588100520 2024-09-19T09:24:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.462+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58819d5e0 con 0x7fd588100520 2024-09-19T09:24:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.462+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c031bb0 con 0x7fd588100520 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.463+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd57c05d060 con 0x7fd588100520 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.463+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588069f90 con 0x7fd588100520 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.463+0000 7fd585ffb640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 0x7fd57803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.463+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd57c05b070 con 0x7fd588100520 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.463+0000 7fd587fff640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 0x7fd57803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:20.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.464+0000 7fd587fff640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 0x7fd57803b900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5740098c0 tx=0x7fd574006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.468+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd57c051030 con 0x7fd588100520 2024-09-19T09:24:20.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.744+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd588100980 con 0x7fd588100520 2024-09-19T09:24:20.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.745+0000 7fd585ffb640 1 -- 172.21.15.78:0/1077889831 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd57c0543b0 con 0x7fd588100520 2024-09-19T09:24:20.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:24:20.746 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","modified":"2024-09-19T09:22:47.619360Z","created":"2024-09-19T09:22:47.619360Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T09:24:20.747 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 msgr2=0x7fd57803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 0x7fd57803b900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5740098c0 tx=0x7fd574006db0 comp rx=0 tx=0).stop 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 msgr2=0x7fd5881a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0045e0 tx=0x7fd57c004610 comp rx=0 tx=0).stop 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 shutdown_connections 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd578039440 0x7fd57803b900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:20.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 --2- 172.21.15.78:0/1077889831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd588100520 0x7fd5881a3420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:20.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 >> 172.21.15.78:0/1077889831 conn(0x7fd588077310 msgr2=0x7fd5880fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:20.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 shutdown_connections 2024-09-19T09:24:20.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:20.748+0000 7fd58f048640 1 -- 172.21.15.78:0/1077889831 wait complete. 2024-09-19T09:24:21.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:20 smithi028 ceph-mon[30972]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:21.534 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:21 smithi078 systemd[1]: Starting Ceph mon.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:24:21.802 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:21 smithi078 podman[35599]: 2024-09-19 09:24:21.642152167 +0000 UTC m=+0.022841622 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:24:21.802 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:21 smithi078 podman[35599]: 2024-09-19 09:24:21.801461113 +0000 UTC m=+0.182150568 container create 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:24:22.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:21 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/1077889831' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:22.052 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 podman[35599]: 2024-09-19 09:24:22.020366704 +0000 UTC m=+0.401056154 container init 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T09:24:22.052 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 podman[35599]: 2024-09-19 09:24:22.02341234 +0000 UTC m=+0.404101793 container start 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T09:24:22.052 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T09:24:22.052 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T09:24:22.052 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pidfile_write: ignore empty --pid-file 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: load: jerasure load: lrc 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: RocksDB version: 7.9.2 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Git sha 0 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: DB SUMMARY 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: DB Session ID: 56AHQFATQF9B9DLDAOD7 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: CURRENT file: CURRENT 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.error_if_exists: 0 2024-09-19T09:24:22.053 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.create_if_missing: 0 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.paranoid_checks: 1 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.env: 0x55a57489f940 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.info_log: 0x55a5759836e0 2024-09-19T09:24:22.054 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.statistics: (nil) 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.use_fsync: 0 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_log_file_size: 0 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_fallocate: 1 2024-09-19T09:24:22.055 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.use_direct_reads: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.db_log_dir: 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.wal_dir: 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T09:24:22.056 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.write_buffer_manager: 0x55a575992dc0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.unordered_write: 0 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T09:24:22.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.row_cache: None 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.wal_filter: None 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.two_write_queues: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.wal_compression: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.atomic_flush: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T09:24:22.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.log_readahead_size: 0 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_background_jobs: 2 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_background_compactions: -1 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_subcompactions: 1 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T09:24:22.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_open_files: -1 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_background_flushes: -1 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Compression algorithms supported: 2024-09-19T09:24:22.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kZSTD supported: 0 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kXpressCompression supported: 0 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kBZip2Compression supported: 0 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kLZ4Compression supported: 1 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kZlibCompression supported: 1 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: kSnappyCompression supported: 1 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T09:24:22.061 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.merge_operator: 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_filter: None 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a575983320) 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: index_type: 0 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: data_block_index_type: 0 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: index_shortening: 1 2024-09-19T09:24:22.062 INFO:journalctl@ceph.mon.c.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: checksum: 4 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: no_block_cache: 0 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_cache: 0x55a5759771f0 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_cache_options: 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: capacity : 536870912 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: num_shard_bits : 4 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: strict_capacity_limit : 0 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_cache_compressed: (nil) 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: persistent_cache: (nil) 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_size: 4096 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_size_deviation: 10 2024-09-19T09:24:22.063 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_restart_interval: 16 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: index_block_restart_interval: 1 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: metadata_block_size: 4096 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: partition_filters: 0 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: use_delta_encoding: 1 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: filter_policy: bloomfilter 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: whole_key_filtering: 1 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: verify_compression: 0 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: format_version: 5 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: enable_index_compression: 1 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: block_align: 0 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: max_auto_readahead_size: 262144 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: prepopulate_block_cache: 0 2024-09-19T09:24:22.064 INFO:journalctl@ceph.mon.c.smithi078.stdout: initial_auto_readahead_size: 8192 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression: NoCompression 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.num_levels: 7 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T09:24:22.065 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:24:22.066 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T09:24:22.067 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T09:24:22.068 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T09:24:22.068 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T09:24:22.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.inplace_update_support: 0 2024-09-19T09:24:22.070 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.bloom_locality: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.max_successive_merges: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.ttl: 2592000 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T09:24:22.071 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enable_blob_files: false 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.min_blob_size: 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T09:24:22.072 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1543f6db-ef6d-4fe1-ac3d-a3d13df3f751 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737862050049, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737862050450, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726737862, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1543f6db-ef6d-4fe1-ac3d-a3d13df3f751", "db_session_id": "56AHQFATQF9B9DLDAOD7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737862050543, "job": 1, "event": "recovery_finished"} 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a5759a4e00 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: DB pointer 0x55a575aa0000 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout: ** DB Stats ** 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:24:22.073 INFO:journalctl@ceph.mon.c.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.074 INFO:journalctl@ceph.mon.c.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:24:22.075 INFO:journalctl@ceph.mon.c.smithi078.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout: Block cache BinnedLRUCache@0x55a5759771f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: using public_addr v2:172.21.15.78:0/0 -> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: starting mon.c rank -1 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:22.076 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(???) e0 preinit fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: e1 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: btime 2024-09-19T09:22:57:723574+0000 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: legacy client fscid: -1 2024-09-19T09:24:22.357 INFO:journalctl@ceph.mon.c.smithi078.stdout: 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout: No filesystems configured 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:24:22.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Added host smithi078 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3636745316' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3636745316' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:22.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.360 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Saving service mon spec with placement smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c;count:3 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: Deploying daemon mon.c on smithi078 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/1077889831' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 ceph-mon[35619]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 bash[35599]: 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 2024-09-19T09:24:22.361 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:22 smithi078 systemd[1]: Started Ceph mon.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:24:22.577 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:24:22.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:24:22.829 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:27.421 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.421 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.422 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T09:24:27.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: monmap epoch 2 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: last_changed 2024-09-19T09:24:22.077868+0000 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: election_strategy: 1 2024-09-19T09:24:27.424 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: fsmap 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: mgrmap e13: x(active, since 39s) 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.425 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:27.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:24:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: mon.a calling monitor election 2024-09-19T09:24:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: mon.c calling monitor election 2024-09-19T09:24:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:27.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:27.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: monmap epoch 2 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: last_changed 2024-09-19T09:24:22.077868+0000 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: min_mon_release 19 (squid) 2024-09-19T09:24:27.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: election_strategy: 1 2024-09-19T09:24:27.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:27.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:27.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: fsmap 2024-09-19T09:24:27.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:27.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: mgrmap e13: x(active, since 39s) 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: overall HEALTH_OK 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:27.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:27.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 -- 172.21.15.78:0/2280820881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98f0003530 msgr2=0x7f98f00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 --2- 172.21.15.78:0/2280820881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98f0003530 0x7f98f00059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99081016a0 tx=0x7f98f802c490 comp rx=0 tx=0).stop 2024-09-19T09:24:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 -- 172.21.15.78:0/2280820881 shutdown_connections 2024-09-19T09:24:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 --2- 172.21.15.78:0/2280820881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98f0003530 0x7f98f00059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 -- 172.21.15.78:0/2280820881 >> 172.21.15.78:0/2280820881 conn(0x7f99080fc220 msgr2=0x7f99080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 -- 172.21.15.78:0/2280820881 shutdown_connections 2024-09-19T09:24:27.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.800+0000 7f990e941640 1 -- 172.21.15.78:0/2280820881 wait complete. 2024-09-19T09:24:27.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.801+0000 7f990e941640 1 Processor -- start 2024-09-19T09:24:27.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.801+0000 7f990e941640 1 -- start start 2024-09-19T09:24:27.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f990e941640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f990e941640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 0x7f9908196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f990e941640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9908108670 con 0x7f99081963f0 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f990e941640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99081084f0 con 0x7f98f0003530 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44712/0 (socket says 172.21.15.78:44712) 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 -- 172.21.15.78:0/1741290137 learned_addr learned my addr 172.21.15.78:0/1741290137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:24:27.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 -- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 msgr2=0x7f9908195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T09:24:27.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 -- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 msgr2=0x7f9908195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:24:27.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:24:27.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f99077fe640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 0x7f9908196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.802+0000 7f9907fff640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:24:27.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99077fe640 1 -- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 msgr2=0x7f9908195eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:24:27.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99077fe640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99077fe640 1 -- 172.21.15.78:0/1741290137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990819af00 con 0x7f99081963f0 2024-09-19T09:24:27.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99077fe640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 0x7f9908196850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b3d0 tx=0x7f98fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f98fc011630 con 0x7f99081963f0 2024-09-19T09:24:27.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc011cb0 con 0x7f99081963f0 2024-09-19T09:24:27.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f98fc010df0 con 0x7f99081963f0 2024-09-19T09:24:27.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990819b1f0 con 0x7f99081963f0 2024-09-19T09:24:27.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.803+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990819b780 con 0x7f99081963f0 2024-09-19T09:24:27.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.804+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f98fc018020 con 0x7f99081963f0 2024-09-19T09:24:27.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.804+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9908101ca0 con 0x7f99081963f0 2024-09-19T09:24:27.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.805+0000 7f99057fa640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 0x7f98f403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.805+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98fc04bfb0 con 0x7f99081963f0 2024-09-19T09:24:27.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.805+0000 7f9907fff640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 0x7f98f403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.805+0000 7f9907fff640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 0x7f98f403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98f8012c10 tx=0x7f98f80323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:27.810+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f98fc01f020 con 0x7f99081963f0 2024-09-19T09:24:28.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.045+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f990819c250 con 0x7f99081963f0 2024-09-19T09:24:28.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.045+0000 7f99057fa640 1 -- 172.21.15.78:0/1741290137 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f98fc018310 con 0x7f99081963f0 2024-09-19T09:24:28.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:24:28.046 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","modified":"2024-09-19T09:24:22.077868Z","created":"2024-09-19T09:22:47.619360Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T09:24:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 2 2024-09-19T09:24:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 msgr2=0x7f98f403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 0x7f98f403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98f8012c10 tx=0x7f98f80323a0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 msgr2=0x7f9908196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 0x7f9908196850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b3d0 tx=0x7f98fc00b890 comp rx=0 tx=0).stop 2024-09-19T09:24:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 shutdown_connections 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f98f4039040 0x7f98f403b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99081963f0 0x7f9908196850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 --2- 172.21.15.78:0/1741290137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98f0003530 0x7f9908195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 >> 172.21.15.78:0/1741290137 conn(0x7f99080fc220 msgr2=0x7f99080fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 shutdown_connections 2024-09-19T09:24:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:28.047+0000 7f990e941640 1 -- 172.21.15.78:0/1741290137 wait complete. 2024-09-19T09:24:28.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:28 smithi078 ceph-mon[35619]: Deploying daemon mon.b on smithi038 2024-09-19T09:24:28.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:28 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/1741290137' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:28.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:28.436 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:28 smithi028 ceph-mon[30972]: Deploying daemon mon.b on smithi038 2024-09-19T09:24:28.436 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:28 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/1741290137' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:28.436 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:29.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:29 smithi028 ceph-mon[30972]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:29 smithi078 ceph-mon[35619]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:29.697 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:24:29.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:24:30.223 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- 172.21.15.78:0/2813552675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 msgr2=0x7f3a9c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2813552675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c102c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009880 tx=0x7f3a8c02a670 comp rx=0 tx=0).stop 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- 172.21.15.78:0/2813552675 shutdown_connections 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2813552675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c102c00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- 172.21.15.78:0/2813552675 >> 172.21.15.78:0/2813552675 conn(0x7f3a9c0fc220 msgr2=0x7f3a9c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- 172.21.15.78:0/2813552675 shutdown_connections 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- 172.21.15.78:0/2813552675 wait complete. 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 Processor -- start 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.609+0000 7f3aa183e640 1 -- start start 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3aa183e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3aa183e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3aa183e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c10bd70 con 0x7f3a9c102820 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3aa183e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c10bbf0 con 0x7f3a9c197970 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44716/0 (socket says 172.21.15.78:44716) 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 -- 172.21.15.78:0/2136449813 learned_addr learned my addr 172.21.15.78:0/2136449813 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 -- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 msgr2=0x7f3a9c19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 -- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 msgr2=0x7f3a9c19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9a7fc640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9affd640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9affd640 1 -- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 msgr2=0x7f3a9c19bd00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:24:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9affd640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.610+0000 7f3a9affd640 1 -- 172.21.15.78:0/2136449813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c197f90 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3a9affd640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c197430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3a8c035070 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c032e40 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3a8c03bb50 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c19c240 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.611+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c19c720 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.612+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3a8c042030 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.612+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9c06ad10 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.612+0000 7f3aa083c640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 0x7f3a8403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.612+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a8c040070 con 0x7f3a9c102820 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.612+0000 7f3a9a7fc640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 0x7f3a8403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.613+0000 7f3a9a7fc640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 0x7f3a8403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009100 tx=0x7f3a90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.617+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a8c03b460 con 0x7f3a9c102820 2024-09-19T09:24:30.794 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:30.794 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:30.794 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:30.794 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: Deploying daemon mon.b on smithi038 2024-09-19T09:24:30.794 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/1741290137' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:30.795 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:30.795 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:30.795 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 ceph-mon[35271]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T09:24:30.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.797+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c003800 con 0x7f3a9c102820 2024-09-19T09:24:30.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:30.849+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3a9c19c9d0 con 0x7f3a9c102820 2024-09-19T09:24:31.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 bash[35252]: 9fa06e52672b41efc7bb05b486efabbfe1e891ff1a0158e787c0bd561b133948 2024-09-19T09:24:31.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:30 smithi038 systemd[1]: Started Ceph mon.b for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:24:35.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.806+0000 7f3aa083c640 1 -- 172.21.15.78:0/2136449813 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f3a8c003b80 con 0x7f3a9c102820 2024-09-19T09:24:35.807 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:24:35.807 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","modified":"2024-09-19T09:24:30.796932Z","created":"2024-09-19T09:22:47.619360Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:3300","nonce":0},{"type":"v1","addr":"172.21.15.28:6789","nonce":0}]},"addr":"172.21.15.28:6789/0","public_addr":"172.21.15.28:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T09:24:35.808 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 3 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 msgr2=0x7f3a8403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 0x7f3a8403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009100 tx=0x7f3a90008040 comp rx=0 tx=0).stop 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 msgr2=0x7f3a9c197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c197430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c032c50 comp rx=0 tx=0).stop 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 shutdown_connections 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3a84039040 0x7f3a8403b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a9c197970 0x7f3a9c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 --2- 172.21.15.78:0/2136449813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a9c102820 0x7f3a9c197430 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 >> 172.21.15.78:0/2136449813 conn(0x7f3a9c0fc220 msgr2=0x7f3a9c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:35.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 shutdown_connections 2024-09-19T09:24:35.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:35.809+0000 7f3aa183e640 1 -- 172.21.15.78:0/2136449813 wait complete. 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: mon.a calling monitor election 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: mon.c calling monitor election 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/2136449813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: monmap epoch 3 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: min_mon_release 19 (squid) 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: election_strategy: 1 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:36.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: fsmap 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: mgrmap e13: x(active, since 48s) 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: overall HEALTH_OK 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:24:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T09:24:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T09:24:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/2136449813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: monmap epoch 3 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:36.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: election_strategy: 1 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: fsmap 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: mgrmap e13: x(active, since 48s) 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T09:24:36.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.267 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.267 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.267 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:36.538 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T09:24:36.538 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T09:24:36.802 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:37.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:36 smithi078 ceph-mon[35619]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:37.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:37.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:37.202 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:36 smithi028 ceph-mon[30972]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:37.202 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:37.202 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 -- 172.21.15.28:0/3934721686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 msgr2=0x7ff6bc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/3934721686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc106920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009810 tx=0x7ff6b002a540 comp rx=0 tx=0).stop 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 -- 172.21.15.28:0/3934721686 shutdown_connections 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/3934721686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc106920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 -- 172.21.15.28:0/3934721686 >> 172.21.15.28:0/3934721686 conn(0x7ff6bc077310 msgr2=0x7ff6bc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:37.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 -- 172.21.15.28:0/3934721686 shutdown_connections 2024-09-19T09:24:37.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.284+0000 7ff6c2e29640 1 -- 172.21.15.28:0/3934721686 wait complete. 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 Processor -- start 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 -- start start 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6bc1a11b0 0x7ff6bc1a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc10ab20 con 0x7ff6bc106540 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc10a9a0 con 0x7ff6bc198680 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c2e29640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc10aca0 con 0x7ff6bc1a11b0 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.285+0000 7ff6c0bc5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:37.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46846/0 (socket says 172.21.15.28:46846) 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 -- 172.21.15.28:0/543252769 learned_addr learned my addr 172.21.15.28:0/543252769 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c13c6640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6bc1a11b0 0x7ff6bc1a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6bbfff640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6bbfff640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 msgr2=0x7ff6bc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6bbfff640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 msgr2=0x7ff6bc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6bbfff640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6bbfff640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6bc1a11b0 msgr2=0x7ff6bc1a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6bc1a11b0 0x7ff6bc1a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 msgr2=0x7ff6bc198ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 -- 172.21.15.28:0/543252769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1a3d40 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c0bc5640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00307a0 tx=0x7ff6b00307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0040070 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b0004c70 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1a3fd0 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.286+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b002f570 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.287+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc1a4480 con 0x7ff6bc106540 2024-09-19T09:24:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff6b0039040 con 0x7ff6bc106540 2024-09-19T09:24:37.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6b9ffb640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 0x7ff6ac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:37.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc101bb0 con 0x7ff6bc106540 2024-09-19T09:24:37.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff6b006d6f0 con 0x7ff6bc106540 2024-09-19T09:24:37.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6bbfff640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 0x7ff6ac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:37.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.288+0000 7ff6bbfff640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 0x7ff6ac03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:37.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.297+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff6b0004e20 con 0x7ff6bc106540 2024-09-19T09:24:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.486+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff6bc1a4780 con 0x7ff6bc106540 2024-09-19T09:24:37.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.486+0000 7ff6b9ffb640 1 -- 172.21.15.28:0/543252769 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7ff6b0043360 con 0x7ff6bc106540 2024-09-19T09:24:37.488 INFO:teuthology.orchestra.run.smithi028.stdout:# minimal ceph.conf for 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:37.488 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-09-19T09:24:37.488 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:37.488 INFO:teuthology.orchestra.run.smithi028.stdout: mon_host = [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] 2024-09-19T09:24:37.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 msgr2=0x7ff6ac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:37.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 0x7ff6ac03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8008040 comp rx=0 tx=0).stop 2024-09-19T09:24:37.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 msgr2=0x7ff6bc198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00307a0 tx=0x7ff6b00307d0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 shutdown_connections 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff6ac039730 0x7ff6ac03bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6bc198680 0x7ff6bc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6bc1a11b0 0x7ff6bc1a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 --2- 172.21.15.28:0/543252769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6bc106540 0x7ff6bc198140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 >> 172.21.15.28:0/543252769 conn(0x7ff6bc077310 msgr2=0x7ff6bc0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:37.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 shutdown_connections 2024-09-19T09:24:37.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:37.489+0000 7ff6c2e29640 1 -- 172.21.15.28:0/543252769 wait complete. 2024-09-19T09:24:38.161 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T09:24:38.161 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:24:38.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: mon.a calling monitor election 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: mon.c calling monitor election 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/2136449813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:38.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: monmap epoch 3 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: min_mon_release 19 (squid) 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: election_strategy: 1 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:38.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: fsmap 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: mgrmap e13: x(active, since 48s) 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: overall HEALTH_OK 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:38.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:38.190 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:24:38.190 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:38.257 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:24:38.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:24:38.285 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:24:38.285 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:38.350 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:24:38.350 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:24:38.378 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:24:38.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:24:38.442 INFO:tasks.cephadm:Adding mgr.x on smithi028 2024-09-19T09:24:38.443 INFO:tasks.cephadm:Adding mgr.y on smithi038 2024-09-19T09:24:38.443 INFO:tasks.cephadm:Adding mgr.z on smithi078 2024-09-19T09:24:38.443 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi028=x;smithi038=y;smithi078=z' 2024-09-19T09:24:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:38.978 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mon.b calling monitor election 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi028:/etc/ceph/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Updating smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mon.b calling monitor election 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mon.c calling monitor election 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mon.a calling monitor election 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:24:38.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: monmap epoch 3 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: election_strategy: 1 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: fsmap 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: mgrmap e13: x(active, since 50s) 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T09:24:38.980 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:38.981 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:38 smithi028 ceph-mon[30972]: Reconfiguring daemon mon.a on smithi028 2024-09-19T09:24:39.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mon.b calling monitor election 2024-09-19T09:24:39.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi028:/etc/ceph/ceph.conf 2024-09-19T09:24:39.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T09:24:39.057 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Updating smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mon.b calling monitor election 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mon.c calling monitor election 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mon.a calling monitor election 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: monmap epoch 3 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:39.058 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: min_mon_release 19 (squid) 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: election_strategy: 1 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: fsmap 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: mgrmap e13: x(active, since 50s) 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: overall HEALTH_OK 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.059 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.060 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:38 smithi078 ceph-mon[35619]: Reconfiguring daemon mon.a on smithi028 2024-09-19T09:24:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 -- 172.21.15.78:0/1878279604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2481005f0 msgr2=0x7fa2481009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 --2- 172.21.15.78:0/1878279604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2481005f0 0x7fa2481009f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009880 tx=0x7fa23c02a5d0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 -- 172.21.15.78:0/1878279604 shutdown_connections 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 --2- 172.21.15.78:0/1878279604 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2481005f0 0x7fa2481009f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 -- 172.21.15.78:0/1878279604 >> 172.21.15.78:0/1878279604 conn(0x7fa2480fbda0 msgr2=0x7fa2480fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 -- 172.21.15.78:0/1878279604 shutdown_connections 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.136+0000 7fa24f096640 1 -- 172.21.15.78:0/1878279604 wait complete. 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 Processor -- start 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 -- start start 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248195300 0x7fa24819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa248195780 0x7fa24819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481055c0 con 0x7fa248195780 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa248105440 con 0x7fa248195300 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24f096640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa248105740 con 0x7fa2481005f0 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36134/0 (socket says 172.21.15.78:36134) 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 -- 172.21.15.78:0/3906589493 learned_addr learned my addr 172.21.15.78:0/3906589493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24d633640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa248195780 0x7fa24819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa247fff640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248195300 0x7fa24819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 -- 172.21.15.78:0/3906589493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248195300 msgr2=0x7fa24819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248195300 0x7fa24819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 -- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa248195780 msgr2=0x7fa24819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa248195780 0x7fa24819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.137+0000 7fa24ce32640 1 -- 172.21.15.78:0/3906589493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24819d1c0 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa24ce32640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa23c031610 tx=0x7fa23c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c03f030 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24819d450 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c003ce0 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c035070 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.138+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24819d9b0 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.139+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa23c042030 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.139+0000 7fa245ffb640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 0x7fa23803de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.139+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa23c06cee0 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.139+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa248100a70 con 0x7fa2481005f0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.139+0000 7fa247fff640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 0x7fa23803de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.140+0000 7fa247fff640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 0x7fa23803de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2481962e0 tx=0x7fa234009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.144+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa23c043d80 con 0x7fa2481005f0 2024-09-19T09:24:39.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mon.b calling monitor election 2024-09-19T09:24:39.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi028:/etc/ceph/ceph.conf 2024-09-19T09:24:39.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T09:24:39.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-19T09:24:39.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Updating smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mon.b calling monitor election 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mon.c calling monitor election 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mon.a calling monitor election 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: monmap epoch 3 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: last_changed 2024-09-19T09:24:30.796932+0000 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: created 2024-09-19T09:22:47.619360+0000 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: min_mon_release 19 (squid) 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: election_strategy: 1 2024-09-19T09:24:39.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: 0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.c 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: 2: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: fsmap 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: mgrmap e13: x(active, since 50s) 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: overall HEALTH_OK 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.178 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:24:39.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:39.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:39.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.181 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:38 smithi038 ceph-mon[35271]: Reconfiguring daemon mon.a on smithi028 2024-09-19T09:24:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.322+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi028=x;smithi038=y;smithi078=z", "target": ["mon-mgr", ""]}) -- 0x7fa2480626c0 con 0x7fa23803b940 2024-09-19T09:24:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.328+0000 7fa245ffb640 1 -- 172.21.15.78:0/3906589493 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa2480626c0 con 0x7fa23803b940 2024-09-19T09:24:39.330 INFO:teuthology.orchestra.run.smithi078.stdout:Scheduled mgr update... 2024-09-19T09:24:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.329+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 msgr2=0x7fa23803de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 0x7fa23803de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2481962e0 tx=0x7fa234009190 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 msgr2=0x7fa248194dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa23c031610 tx=0x7fa23c0043c0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 shutdown_connections 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa23803b940 0x7fa23803de00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248195300 0x7fa24819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa248195780 0x7fa24819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 --2- 172.21.15.78:0/3906589493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481005f0 0x7fa248194dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 >> 172.21.15.78:0/3906589493 conn(0x7fa2480fbda0 msgr2=0x7fa2480fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 shutdown_connections 2024-09-19T09:24:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:24:39.330+0000 7fa24f096640 1 -- 172.21.15.78:0/3906589493 wait complete. 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: Reconfiguring daemon mon.b on smithi038 2024-09-19T09:24:39.902 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: Reconfiguring daemon mon.b on smithi038 2024-09-19T09:24:39.903 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:39.930 DEBUG:teuthology.orchestra.run.smithi038:mgr.y> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T09:24:39.933 DEBUG:teuthology.orchestra.run.smithi078:mgr.z> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T09:24:39.935 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T09:24:39.936 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:24:39.936 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:24:39.957 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:24:39.957 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 09:23:57.951376963 +0000 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 09:19:27.570575915 +0000 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 09:19:27.570575915 +0000 2024-09-19T09:24:40.017 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 09:19:27.570575915 +0000 2024-09-19T09:24:40.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:24:40.084 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:24:40.084 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:40.084 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:40.084 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:24:40.084 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:40.085 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:40.085 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:40.085 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: Reconfiguring daemon mon.b on smithi038 2024-09-19T09:24:40.085 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:40.087 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T09:24:40.088 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T09:24:40.088 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000127285 s, 4.0 MB/s 2024-09-19T09:24:40.089 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:24:40.149 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-09-19T09:24:40.208 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 09:23:57.978376224 +0000 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 09:19:27.888567152 +0000 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 09:19:27.888567152 +0000 2024-09-19T09:24:40.209 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 09:19:27.888567152 +0000 2024-09-19T09:24:40.209 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:24:40.277 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T09:24:40.277 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T09:24:40.277 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000126939 s, 4.0 MB/s 2024-09-19T09:24:40.279 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:24:40.339 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-09-19T09:24:40.397 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:24:40.397 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 09:23:58.007375430 +0000 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 09:19:28.205558418 +0000 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 09:19:28.205558418 +0000 2024-09-19T09:24:40.398 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 09:19:28.205558418 +0000 2024-09-19T09:24:40.398 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:24:40.465 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T09:24:40.465 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T09:24:40.465 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000133077 s, 3.8 MB/s 2024-09-19T09:24:40.466 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:24:40.526 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-09-19T09:24:40.584 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:24:40.584 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 09:23:58.034374690 +0000 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 09:19:28.520549738 +0000 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 09:19:28.520549738 +0000 2024-09-19T09:24:40.585 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 09:19:28.520549738 +0000 2024-09-19T09:24:40.586 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:24:40.651 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T09:24:40.651 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T09:24:40.651 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000118721 s, 4.3 MB/s 2024-09-19T09:24:40.652 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:24:40.710 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T09:24:40.711 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:24:40.728 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:24:40.728 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 09:24:30.244691903 +0000 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 09:19:42.170836445 +0000 2024-09-19T09:24:40.784 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 09:19:42.170836445 +0000 2024-09-19T09:24:40.785 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 09:19:42.170836445 +0000 2024-09-19T09:24:40.785 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:24:40.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T09:24:40.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T09:24:40.849 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000277508 s, 1.8 MB/s 2024-09-19T09:24:40.850 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:24:40.907 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-09-19T09:24:40.963 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 09:24:30.245691882 +0000 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 09:19:42.519829285 +0000 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 09:19:42.519829285 +0000 2024-09-19T09:24:40.964 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 09:19:42.519829285 +0000 2024-09-19T09:24:40.965 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:24:41.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T09:24:41.028 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T09:24:41.028 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000278196 s, 1.8 MB/s 2024-09-19T09:24:41.029 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:24:41.085 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-09-19T09:24:41.140 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:24:41.140 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:41.140 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 09:24:30.245691882 +0000 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 09:19:42.849822515 +0000 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 09:19:42.849822515 +0000 2024-09-19T09:24:41.141 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 09:19:42.849822515 +0000 2024-09-19T09:24:41.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:24:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T09:24:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T09:24:41.204 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000273339 s, 1.9 MB/s 2024-09-19T09:24:41.205 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:24:41.262 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 09:24:30.245691882 +0000 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 09:19:43.169815950 +0000 2024-09-19T09:24:41.317 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 09:19:43.169815950 +0000 2024-09-19T09:24:41.318 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 09:19:43.169815950 +0000 2024-09-19T09:24:41.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:24:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T09:24:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T09:24:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000258825 s, 2.0 MB/s 2024-09-19T09:24:41.381 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:24:41.437 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-19T09:24:41.437 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:24:41.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:24:41.455 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-09-19T09:24:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:24:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-19 09:24:21.285239140 +0000 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-19 09:19:36.998311588 +0000 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-19 09:19:36.998311588 +0000 2024-09-19T09:24:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-19 09:19:36.998311588 +0000 2024-09-19T09:24:41.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:24:41.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:41.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi028=x;smithi038=y;smithi078=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:41.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: Saving service mgr spec with placement smithi028=x;smithi038=y;smithi078=z;count:3 2024-09-19T09:24:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:41.573 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:41.573 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi028=x;smithi038=y;smithi078=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:41.573 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: Saving service mgr spec with placement smithi028=x;smithi038=y;smithi078=z;count:3 2024-09-19T09:24:41.574 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.574 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.574 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:41.574 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:41.574 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:41.576 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-19T09:24:41.576 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-19T09:24:41.576 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000342602 s, 1.5 MB/s 2024-09-19T09:24:41.578 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:24:41.634 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-09-19T09:24:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi028=x;smithi038=y;smithi078=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: Saving service mgr spec with placement smithi028=x;smithi038=y;smithi078=z;count:3 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:24:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-19 09:24:21.286239126 +0000 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-19 09:19:37.323306743 +0000 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-19 09:19:37.323306743 +0000 2024-09-19T09:24:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-19 09:19:37.323306743 +0000 2024-09-19T09:24:41.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:24:41.752 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-19T09:24:41.752 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-19T09:24:41.752 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000206428 s, 2.5 MB/s 2024-09-19T09:24:41.753 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:24:41.810 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-19 09:24:21.286239126 +0000 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-19 09:19:37.678301451 +0000 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-19 09:19:37.678301451 +0000 2024-09-19T09:24:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-19 09:19:37.678301451 +0000 2024-09-19T09:24:41.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:24:41.930 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-19T09:24:41.931 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-19T09:24:41.931 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000192637 s, 2.7 MB/s 2024-09-19T09:24:41.932 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:24:41.987 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-19 09:24:21.286239126 +0000 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-19 09:19:37.995296726 +0000 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-19 09:19:37.995296726 +0000 2024-09-19T09:24:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-19 09:19:37.995296726 +0000 2024-09-19T09:24:42.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:24:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-19T09:24:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-19T09:24:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000197019 s, 2.6 MB/s 2024-09-19T09:24:42.108 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:24:42.164 INFO:tasks.cephadm:Deploying osd.0 on smithi028 with /dev/vg_nvme/lv_4... 2024-09-19T09:24:42.164 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:24:42.428 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:42.487 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:24:42.487 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: Reconfiguring daemon mon.c on smithi078 2024-09-19T09:24:42.487 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.487 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:42.488 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:42.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: Reconfiguring daemon mon.c on smithi078 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:42.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:42.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:42.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:42.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: Reconfiguring daemon mon.c on smithi078 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:43 smithi028 ceph-mon[30972]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:43 smithi028 ceph-mon[30972]: Deploying daemon mgr.z on smithi078 2024-09-19T09:24:43.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:43 smithi078 ceph-mon[35619]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:43.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:43 smithi078 ceph-mon[35619]: Deploying daemon mgr.z on smithi078 2024-09-19T09:24:43.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:43 smithi038 ceph-mon[35271]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:43.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:43 smithi038 ceph-mon[35271]: Deploying daemon mgr.z on smithi078 2024-09-19T09:24:43.883 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:43 smithi078 podman[37306]: 2024-09-19 09:24:43.882913535 +0000 UTC m=+0.157228581 container create d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:24:44.170 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 podman[37306]: 2024-09-19 09:24:44.127749383 +0000 UTC m=+0.402064383 container init d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:24:44.171 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 podman[37306]: 2024-09-19 09:24:44.130672139 +0000 UTC m=+0.404987147 container start d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-mgr[37325]: -- 172.21.15.78:0/218852544 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b03f1930e0 con 0x55b03f171000 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 bash[37306]: d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 systemd[1]: Started Ceph mgr.z for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.250+0000 7fcdc5d8d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.310+0000 7fcdc5d8d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:24:44.459 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.456+0000 7fcdc5d8d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:24:44.798 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:24:44.808 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.522+0000 7fcdc5d8d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:24:44.809 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.667+0000 7fcdc5d8d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:24:44.809 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:44.806+0000 7fcdc5d8d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:24:44.825 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_4 2024-09-19T09:24:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:45.278 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:45.278 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.278 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.278 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.279 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:45.279 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:45.279 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:45.279 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:45.490 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:45.490 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.490 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.490 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.490 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:45.491 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:45.491 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:45.491 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:45.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 -- 172.21.15.28:0/163676842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c100ad0 msgr2=0x7ff50c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 --2- 172.21.15.28:0/163676842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c100ad0 0x7ff50c100ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff50002a410 comp rx=0 tx=0).stop 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 -- 172.21.15.28:0/163676842 shutdown_connections 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 --2- 172.21.15.28:0/163676842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 0x7ff50c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 --2- 172.21.15.28:0/163676842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c102690 0x7ff50c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 --2- 172.21.15.28:0/163676842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c100ad0 0x7ff50c100ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 -- 172.21.15.28:0/163676842 >> 172.21.15.28:0/163676842 conn(0x7ff50c0fc280 msgr2=0x7ff50c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 -- 172.21.15.28:0/163676842 shutdown_connections 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.499+0000 7ff512ea2640 1 -- 172.21.15.28:0/163676842 wait complete. 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 Processor -- start 2024-09-19T09:24:45.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 -- start start 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c100ad0 0x7ff50c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 0x7ff50c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10c730 con 0x7ff50c102690 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10c5b0 con 0x7ff50c101cd0 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff512ea2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10c8b0 con 0x7ff50c100ad0 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42420/0 (socket says 172.21.15.28:42420) 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 -- 172.21.15.28:0/2428147390 learned_addr learned my addr 172.21.15.28:0/2428147390 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:24:45.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff510c3e640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c100ad0 0x7ff50c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff50bfff640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 0x7ff50c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 -- 172.21.15.28:0/2428147390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c100ad0 msgr2=0x7ff50c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c100ad0 0x7ff50c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 -- 172.21.15.28:0/2428147390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 msgr2=0x7ff50c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 0x7ff50c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 -- 172.21.15.28:0/2428147390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c19fb50 con 0x7ff50c102690 2024-09-19T09:24:45.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.500+0000 7ff51143f640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff50400eaa0 tx=0x7ff50400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.501+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400cdb0 con 0x7ff50c102690 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.501+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff504005020 con 0x7ff50c102690 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.501+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c19fe40 con 0x7ff50c102690 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.501+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040116b0 con 0x7ff50c102690 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.501+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1a02f0 con 0x7ff50c102690 2024-09-19T09:24:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff504011850 con 0x7ff50c102690 2024-09-19T09:24:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff509ffb640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 0x7ff4fc03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c1a05b0 con 0x7ff50c102690 2024-09-19T09:24:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff50404c970 con 0x7ff50c102690 2024-09-19T09:24:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff510c3e640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 0x7ff4fc03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.502+0000 7ff510c3e640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 0x7ff4fc03b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff500005ab0 tx=0x7ff500000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:45.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.507+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff504016030 con 0x7ff50c102690 2024-09-19T09:24:45.606 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:45.115+0000 7fcdc5d8d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:24:45.607 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:45.175+0000 7fcdc5d8d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:24:45.607 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:45.359+0000 7fcdc5d8d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:24:45.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:45.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:45.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:45.689+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff50c06a750 con 0x7ff4fc039500 2024-09-19T09:24:46.241 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 podman[36619]: 2024-09-19 09:24:46.070462832 +0000 UTC m=+0.015466054 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:24:46.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:46 smithi028 ceph-mon[30972]: Deploying daemon mgr.y on smithi038 2024-09-19T09:24:46.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:46.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:46.426 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:46.505 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-mon[35271]: Deploying daemon mgr.y on smithi038 2024-09-19T09:24:46.505 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:46.505 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:46.505 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:46.506 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 podman[36619]: 2024-09-19 09:24:46.366120194 +0000 UTC m=+0.311123403 container init 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T09:24:46.506 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 podman[36619]: 2024-09-19 09:24:46.369005774 +0000 UTC m=+0.314008982 container start 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:24:46.506 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-mgr[36638]: -- 172.21.15.38:0/1603874237 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e172bd0e0 con 0x560e1729b000 2024-09-19T09:24:46.506 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 bash[36619]: 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 2024-09-19T09:24:46.506 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 systemd[1]: Started Ceph mgr.y for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:24:46.606 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:46.236+0000 7fcdc5d8d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:24:46.606 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:46.303+0000 7fcdc5d8d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:24:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-mon[35619]: Deploying daemon mgr.y on smithi038 2024-09-19T09:24:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:46.800 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:46.504+0000 7f648eb9b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:24:46.800 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:46.570+0000 7f648eb9b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:24:46.800 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:46.728+0000 7f648eb9b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:24:46.800 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:46.799+0000 7f648eb9b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:24:47.118 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:46.964+0000 7f648eb9b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.063+0000 7fcdc5d8d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.178+0000 7fcdc5d8d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.326+0000 7fcdc5d8d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:47.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.330 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.330 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.330 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.330 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.378 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:47.378 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:47.116+0000 7f648eb9b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.606 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.390+0000 7fcdc5d8d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:24:47.606 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.542+0000 7fcdc5d8d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:24:47.675 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:47.453+0000 7f648eb9b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:24:47.675 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:47.516+0000 7f648eb9b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:24:47.910 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.611+0000 7fcdc5d8d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:24:47.910 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.671+0000 7fcdc5d8d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:24:47.910 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.840+0000 7fcdc5d8d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:24:47.911 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.908+0000 7fcdc5d8d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:24:48.175 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:47.718+0000 7f648eb9b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:24:48.178 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:47.975+0000 7fcdc5d8d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:24:48.178 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:48.057+0000 7fcdc5d8d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:24:48.178 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:48.117+0000 7fcdc5d8d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:24:48.178 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:48.176+0000 7fcdc5d8d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:24:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: Reconfiguring daemon mgr.x on smithi028 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]: dispatch 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]': finished 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:24:48.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:24:48.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:48.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: Reconfiguring daemon mgr.x on smithi028 2024-09-19T09:24:48.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]: dispatch 2024-09-19T09:24:48.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]': finished 2024-09-19T09:24:48.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:24:48.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.642 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: Reconfiguring daemon mgr.x on smithi028 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]: dispatch 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/441275085' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8eb04994-c2e4-4428-8ec0-0a9597ab9498"}]': finished 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:24:48.643 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:24:48.925 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:48.640+0000 7f648eb9b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:24:48.925 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:48.710+0000 7f648eb9b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:24:49.353 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:24:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z[37321]: 2024-09-19T09:24:48.963+0000 7fcdc5d8d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:24:49.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:49.355+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7ff5040108f0 con 0x7ff50c102690 2024-09-19T09:24:49.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1467084122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:49.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: Standby manager daemon z started 2024-09-19T09:24:49.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:24:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:24:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:49 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:49.675 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:49.460+0000 7f648eb9b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:24:49.675 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:49.580+0000 7f648eb9b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1467084122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: Standby manager daemon z started 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:24:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:49.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1467084122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: Standby manager daemon z started 2024-09-19T09:24:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:24:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:24:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:49 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.78:0/1049455411' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:50.019 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:49.729+0000 7f648eb9b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:24:50.020 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:49.792+0000 7f648eb9b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:24:50.020 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:49.948+0000 7f648eb9b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:24:50.020 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.018+0000 7f648eb9b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:24:50.313 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.080+0000 7f648eb9b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:24:50.313 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.244+0000 7f648eb9b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:24:50.313 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.311+0000 7f648eb9b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:24:50.590 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-mon[35271]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-mon[35271]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.381+0000 7f648eb9b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.466+0000 7f648eb9b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.528+0000 7f648eb9b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:24:50.591 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:50.589+0000 7f648eb9b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:24:50.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:50 smithi078 ceph-mon[35619]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:50.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:50 smithi078 ceph-mon[35619]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T09:24:50.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:24:50.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:50 smithi028 ceph-mon[30972]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:50.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:50 smithi028 ceph-mon[30972]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T09:24:50.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:24:51.675 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:24:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y[36634]: 2024-09-19T09:24:51.413+0000 7f648eb9b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:24:52.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:24:52.366+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff504004b20 con 0x7ff50c102690 2024-09-19T09:24:52.617 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:52.618 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: Standby manager daemon y started 2024-09-19T09:24:52.618 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:24:52.618 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:52.618 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:24:52.618 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:52 smithi028 ceph-mon[30972]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:52.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: Standby manager daemon y started 2024-09-19T09:24:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:24:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:24:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:52 smithi038 ceph-mon[35271]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: Standby manager daemon y started 2024-09-19T09:24:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:24:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:24:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:24:52.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:52 smithi078 ceph-mon[35619]: from='mgr.? 172.21.15.38:0/686036917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:24:53.489 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:53 smithi028 ceph-mon[30972]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-19T09:24:53.489 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:24:53.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:53 smithi038 ceph-mon[35271]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-19T09:24:53.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:24:53.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:53 smithi078 ceph-mon[35619]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-19T09:24:53.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:24:54.512 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:54 smithi028 ceph-mon[30972]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:54.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:54 smithi038 ceph-mon[35271]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:54.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:54 smithi078 ceph-mon[35619]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:55.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:55.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:55.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:55.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:55.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:24:57.100 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:57 smithi028 ceph-mon[30972]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:57.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:57 smithi078 ceph-mon[35619]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:57.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:57 smithi038 ceph-mon[35271]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:58.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:24:58.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:58.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:58 smithi028 ceph-mon[30972]: Deploying daemon osd.0 on smithi028 2024-09-19T09:24:58.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:24:58.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:58.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:58 smithi078 ceph-mon[35619]: Deploying daemon osd.0 on smithi028 2024-09-19T09:24:58.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:24:58.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:58.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:58 smithi038 ceph-mon[35271]: Deploying daemon osd.0 on smithi028 2024-09-19T09:24:59.049 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:24:59 smithi028 ceph-mon[30972]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:59.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:24:59 smithi078 ceph-mon[35619]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:24:59.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:24:59 smithi038 ceph-mon[35271]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:01.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:01 smithi078 ceph-mon[35619]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:01.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:01 smithi038 ceph-mon[35271]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:01.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:01 smithi028 ceph-mon[30972]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:03.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:03 smithi028 ceph-mon[30972]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:03.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:03 smithi078 ceph-mon[35619]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:03.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:03 smithi038 ceph-mon[35271]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:04.261 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:04.261 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:04.261 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:04.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:04.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:04.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:04.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:04.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:04.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:05.135 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:05 smithi028 ceph-mon[30972]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:05.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:05 smithi078 ceph-mon[35619]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:05.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:05 smithi038 ceph-mon[35271]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:07.055 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 0 on host 'smithi028' 2024-09-19T09:25:07.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.053+0000 7ff509ffb640 1 -- 172.21.15.28:0/2428147390 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff50c06a750 con 0x7ff4fc039500 2024-09-19T09:25:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.055+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 msgr2=0x7ff4fc03b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.055+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 0x7ff4fc03b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff500005ab0 tx=0x7ff500000bc0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.055+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 msgr2=0x7ff50c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.055+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff50400eaa0 tx=0x7ff50400ef60 comp rx=0 tx=0).stop 2024-09-19T09:25:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.055+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 shutdown_connections 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4fc039500 0x7ff4fc03b9c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff50c101cd0 0x7ff50c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff50c102690 0x7ff50c19f4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 --2- 172.21.15.28:0/2428147390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c100ad0 0x7ff50c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 >> 172.21.15.28:0/2428147390 conn(0x7ff50c0fc280 msgr2=0x7ff50c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 shutdown_connections 2024-09-19T09:25:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:25:07.056+0000 7ff512ea2640 1 -- 172.21.15.28:0/2428147390 wait complete. 2024-09-19T09:25:07.360 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.361 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:07.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:07.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:07.657 DEBUG:teuthology.orchestra.run.smithi028:osd.0> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T09:25:07.659 INFO:tasks.cephadm:Deploying osd.1 on smithi038 with /dev/vg_nvme/lv_4... 2024-09-19T09:25:07.659 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:25:07.904 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:25:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:09 smithi028 ceph-mon[30972]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:09.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:09 smithi078 ceph-mon[35619]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:09.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:09 smithi038 ceph-mon[35271]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:10.014 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:25:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:25:09.756+0000 7f8666a66640 -1 osd.0 0 log_to_monitors true 2024-09-19T09:25:10.101 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:25:10.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-09-19T09:25:10.333 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:10 smithi028 ceph-mon[30972]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:25:10.393 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:25:10.417 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:10 smithi038 ceph-mon[35271]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:25:10.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:10 smithi078 ceph-mon[35619]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:25:10.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- 172.21.15.38:0/918831346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 msgr2=0x7fb70c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- 172.21.15.38:0/918831346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c102000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb708009900 tx=0x7fb70802a410 comp rx=0 tx=0).stop 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- 172.21.15.38:0/918831346 shutdown_connections 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- 172.21.15.38:0/918831346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c1037c0 0x7fb70c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- 172.21.15.38:0/918831346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c102e00 0x7fb70c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- 172.21.15.38:0/918831346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c102000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- 172.21.15.38:0/918831346 >> 172.21.15.38:0/918831346 conn(0x7fb70c0fd390 msgr2=0x7fb70c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- 172.21.15.38:0/918831346 shutdown_connections 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- 172.21.15.38:0/918831346 wait complete. 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 Processor -- start 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 -- start start 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c102e00 0x7fb70c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.862+0000 7fb713cc2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c1037c0 0x7fb70c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb713cc2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10d930 con 0x7fb70c1037c0 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb713cc2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10d7b0 con 0x7fb70c101c00 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb713cc2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10dab0 con 0x7fb70c102e00 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41700/0 (socket says 172.21.15.38:41700) 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 -- 172.21.15.38:0/1987641108 learned_addr learned my addr 172.21.15.38:0/1987641108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb71225f640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c1037c0 0x7fb70c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb71125d640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c102e00 0x7fb70c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 -- 172.21.15.38:0/1987641108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c102e00 msgr2=0x7fb70c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c102e00 0x7fb70c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 -- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c1037c0 msgr2=0x7fb70c19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c1037c0 0x7fb70c19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 -- 172.21.15.38:0/1987641108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c19fbd0 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb711a5e640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb708031440 tx=0x7fb708032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70803f030 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003ce0 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c19fe60 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708035070 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.863+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a02d0 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.865+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c109000 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.866+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fb708042030 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.866+0000 7fb6feffd640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 0x7fb6f80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.866+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7fb7080edf90 con 0x7fb70c101c00 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.866+0000 7fb71125d640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 0x7fb6f80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.867+0000 7fb71125d640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 0x7fb6f80a56a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb70c1994c0 tx=0x7fb700005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:10.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:10.875+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7080bc810 con 0x7fb70c101c00 2024-09-19T09:25:11.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:11.058+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb70c062630 con 0x7fb6f80a31e0 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:11.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.464 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:25:11.129+0000 7f86621c6640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T09:25:11.464 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:25:11.132+0000 7f865e800640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:25:11.464 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:11.464 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:25:11.464 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:25:11.464 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:25:11.464 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:11.465 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:11.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:12.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi028 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi028 to 17092M 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi028 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi028 to 17092M 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi028 2024-09-19T09:25:12.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi028 to 17092M 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: from='osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:13.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:25:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:25:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] boot 2024-09-19T09:25:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:25:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] boot 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:25:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:13.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:25:13.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:25:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:25:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: osd.0 [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] boot 2024-09-19T09:25:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:25:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:25:14.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:25:14.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/960576221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]': finished 2024-09-19T09:25:14.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:25:14.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:14.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/3886915841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/960576221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]': finished 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:14.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/3886915841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/960576221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]: dispatch 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1e8a2f4-f1cf-48d0-af7d-f0e552676eec"}]': finished 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:25:14.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:14.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/3886915841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:15 smithi028 ceph-mon[30972]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:15.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:15 smithi078 ceph-mon[35619]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:15.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:15 smithi038 ceph-mon[35271]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:17.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:17 smithi028 ceph-mon[30972]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:17.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:17 smithi078 ceph-mon[35619]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:17.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:17 smithi038 ceph-mon[35271]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:19.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:19 smithi078 ceph-mon[35619]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:19.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:19 smithi038 ceph-mon[35271]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:19.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:19 smithi028 ceph-mon[30972]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:21 smithi038 ceph-mon[35271]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:21.539 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:21 smithi028 ceph-mon[30972]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:21 smithi078 ceph-mon[35619]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:23.597 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:23 smithi038 ceph-mon[35271]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:23.598 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:23 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:25:23.598 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:23 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:23 smithi078 ceph-mon[35619]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:23 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:25:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:23 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:23.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:23 smithi028 ceph-mon[30972]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:23 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:25:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:23 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:24.587 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:24 smithi038 ceph-mon[35271]: Deploying daemon osd.1 on smithi038 2024-09-19T09:25:24.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:24 smithi078 ceph-mon[35619]: Deploying daemon osd.1 on smithi038 2024-09-19T09:25:24.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:24 smithi028 ceph-mon[30972]: Deploying daemon osd.1 on smithi038 2024-09-19T09:25:25.565 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:25 smithi038 ceph-mon[35271]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:25.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:25 smithi078 ceph-mon[35619]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:25 smithi028 ceph-mon[30972]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:27.566 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:27 smithi028 ceph-mon[30972]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:27.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:27 smithi078 ceph-mon[35619]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:27 smithi038 ceph-mon[35271]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:29.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:29 smithi038 ceph-mon[35271]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:29 smithi078 ceph-mon[35619]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:29.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:29 smithi028 ceph-mon[30972]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:31.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:31 smithi078 ceph-mon[35619]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:31.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:31 smithi038 ceph-mon[35271]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:31.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:31 smithi028 ceph-mon[30972]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:32.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:32.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:32.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:32.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:32.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.373 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 1 on host 'smithi038' 2024-09-19T09:25:33.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.372+0000 7fb6feffd640 1 -- 172.21.15.38:0/1987641108 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb70c062630 con 0x7fb6f80a31e0 2024-09-19T09:25:33.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 msgr2=0x7fb6f80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 0x7fb6f80a56a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb70c1994c0 tx=0x7fb700005f90 comp rx=0 tx=0).stop 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 msgr2=0x7fb70c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb708031440 tx=0x7fb708032ce0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 shutdown_connections 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6f80a31e0 0x7fb6f80a56a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb70c1037c0 0x7fb70c19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb70c102e00 0x7fb70c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 --2- 172.21.15.38:0/1987641108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c101c00 0x7fb70c197fa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.374+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 >> 172.21.15.38:0/1987641108 conn(0x7fb70c0fd390 msgr2=0x7fb70c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.375+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 shutdown_connections 2024-09-19T09:25:33.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:25:33.375+0000 7fb713cc2640 1 -- 172.21.15.38:0/1987641108 wait complete. 2024-09-19T09:25:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:33 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:33 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:33 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:33 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:33.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:33 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:33.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:33 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:34.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:33 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:34.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:33 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:34.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:33 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:34.036 DEBUG:teuthology.orchestra.run.smithi038:osd.1> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T09:25:34.038 INFO:tasks.cephadm:Deploying osd.2 on smithi078 with /dev/vg_nvme/lv_4... 2024-09-19T09:25:34.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:25:34.288 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:25:34.723 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:34 smithi078 ceph-mon[35619]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:34 smithi038 ceph-mon[35271]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:34 smithi028 ceph-mon[30972]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:35.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:35 smithi038 ceph-mon[35271]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:35 smithi038 ceph-mon[35271]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:35.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:25:35 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:25:35.499+0000 7fdd15bb0640 -1 osd.1 0 log_to_monitors true 2024-09-19T09:25:35.778 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:35 smithi078 ceph-mon[35619]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:35.778 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:35 smithi078 ceph-mon[35619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:36.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:35 smithi028 ceph-mon[30972]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:36.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:35 smithi028 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:25:36.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:25:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:25:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-mon[35271]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:36.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:25:36.627+0000 7fdd11310640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T09:25:36.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:25:36 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:25:36.630+0000 7fdd0d94a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:36.967 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:36 smithi078 ceph-mon[35619]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:36.970 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:25:36.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-09-19T09:25:37.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:25:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: from='osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:25:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:36 smithi028 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:25:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:25:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 -- 172.21.15.78:0/1600425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f81078c0 msgr2=0x7fd9f8109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/1600425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f81078c0 0x7fd9f8109d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009ca0 tx=0x7fd9f002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 -- 172.21.15.78:0/1600425456 shutdown_connections 2024-09-19T09:25:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/1600425456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f81078c0 0x7fd9f8109d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/1600425456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f8104f90 0x7fd9f8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/1600425456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8102660 0x7fd9f8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 -- 172.21.15.78:0/1600425456 >> 172.21.15.78:0/1600425456 conn(0x7fd9f80fc280 msgr2=0x7fd9f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 -- 172.21.15.78:0/1600425456 shutdown_connections 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.671+0000 7fd9fd83d640 1 -- 172.21.15.78:0/1600425456 wait complete. 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.672+0000 7fd9fd83d640 1 Processor -- start 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.672+0000 7fd9fd83d640 1 -- start start 2024-09-19T09:25:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.672+0000 7fd9fd83d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 0x7fd9f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 0x7fd9f819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810d880 con 0x7fd9f81078c0 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810d700 con 0x7fd9f8104f90 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810da00 con 0x7fd9f8102660 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55040/0 (socket says 172.21.15.78:55040) 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 -- 172.21.15.78:0/2235048849 learned_addr learned my addr 172.21.15.78:0/2235048849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:25:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f67fc640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 0x7fd9f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f77fe640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 0x7fd9f819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 -- 172.21.15.78:0/2235048849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 msgr2=0x7fd9f8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 0x7fd9f8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 -- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 msgr2=0x7fd9f819f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 0x7fd9f819f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 -- 172.21.15.78:0/2235048849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f819fb00 con 0x7fd9f8102660 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f77fe640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 0x7fd9f819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f67fc640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 0x7fd9f8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9f6ffd640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c970 tx=0x7fd9e800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8007bb0 con 0x7fd9f8102660 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e8007d50 con 0x7fd9f8102660 2024-09-19T09:25:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f819fdf0 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800f450 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.673+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a01e0 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.674+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80fdd70 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.675+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fd9e8004fe0 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.676+0000 7fd9fc83b640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 0x7fd9e00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.676+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fd9e80cdec0 con 0x7fd9f8102660 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.676+0000 7fd9f67fc640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 0x7fd9e00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.677+0000 7fd9f67fc640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 0x7fd9e00a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9f81993f0 tx=0x7fd9ec00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.680+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9e8014030 con 0x7fd9f8102660 2024-09-19T09:25:37.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:37.853+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd9f8062630 con 0x7fd9e00a31e0 2024-09-19T09:25:37.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:25:37.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi038 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi038 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T09:25:38.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:38.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:38.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:25:38.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:25:38.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi038 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:38.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:25:38.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] boot 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:38.924 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:25:38.924 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:25:38.924 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:38.924 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] boot 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:38.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: osd.1 [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] boot 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:25:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:40.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:40 smithi078 ceph-mon[35619]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:25:40.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:40 smithi038 ceph-mon[35271]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:25:40.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:40 smithi028 ceph-mon[30972]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:25:41.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/2064354415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]': finished 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:25:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:41 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/942838640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/2064354415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]': finished 2024-09-19T09:25:41.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:25:41.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:25:41.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:41 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/942838640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/2064354415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]: dispatch 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae120b16-9641-4ffb-96b8-14da465baef8"}]': finished 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:25:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:41 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/942838640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:25:43.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:43 smithi078 ceph-mon[35619]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:43.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:43 smithi038 ceph-mon[35271]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:43 smithi028 ceph-mon[30972]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:45.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:45 smithi078 ceph-mon[35619]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:45.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:45 smithi038 ceph-mon[35271]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:45.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:45 smithi028 ceph-mon[30972]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:47.282 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:47 smithi078 ceph-mon[35619]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:47.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:47 smithi038 ceph-mon[35271]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:47 smithi028 ceph-mon[30972]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:49.304 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:49 smithi078 ceph-mon[35619]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:49.304 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:25:49.304 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:49 smithi038 ceph-mon[35271]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:25:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:49 smithi028 ceph-mon[30972]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:25:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:50.127 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:50 smithi078 ceph-mon[35619]: Deploying daemon osd.2 on smithi078 2024-09-19T09:25:50.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:50 smithi038 ceph-mon[35271]: Deploying daemon osd.2 on smithi078 2024-09-19T09:25:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:50 smithi028 ceph-mon[30972]: Deploying daemon osd.2 on smithi078 2024-09-19T09:25:51.347 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:51 smithi078 ceph-mon[35619]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:51.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:51 smithi038 ceph-mon[35271]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:51.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:51 smithi028 ceph-mon[30972]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:53.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:53 smithi078 ceph-mon[35619]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:53.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:53 smithi038 ceph-mon[35271]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:53.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:53 smithi028 ceph-mon[30972]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:55.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:55 smithi078 ceph-mon[35619]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:55.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:55 smithi038 ceph-mon[35271]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:55.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:55 smithi028 ceph-mon[30972]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:56.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:56 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:56.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:56 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:56.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:56 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:56.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:56 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:56.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:56 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:56.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:56 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:56.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:56 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:56.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:56 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:56.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:56 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:57.404 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:57 smithi078 ceph-mon[35619]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:57.415 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:57 smithi028 ceph-mon[30972]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:57.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:57 smithi038 ceph-mon[35271]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:59.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:59.114 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.712+0000 7fd9fc83b640 1 -- 172.21.15.78:0/2235048849 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd9f8062630 con 0x7fd9e00a31e0 2024-09-19T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 2 on host 'smithi078' 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 msgr2=0x7fd9e00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 0x7fd9e00a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9f81993f0 tx=0x7fd9ec00b5f0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 msgr2=0x7fd9f8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c970 tx=0x7fd9e800ce30 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 shutdown_connections 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9e00a31e0 0x7fd9e00a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9f8104f90 0x7fd9f8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9f81078c0 0x7fd9f819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 --2- 172.21.15.78:0/2235048849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9f8102660 0x7fd9f8197ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 >> 172.21.15.78:0/2235048849 conn(0x7fd9f80fc280 msgr2=0x7fd9f8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 shutdown_connections 2024-09-19T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:25:59.715+0000 7fd9fd83d640 1 -- 172.21.15.78:0/2235048849 wait complete. 2024-09-19T09:26:00.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.069 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:25:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:25:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:25:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:00.410 DEBUG:teuthology.orchestra.run.smithi078:osd.2> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T09:26:00.412 INFO:tasks.cephadm:Deploying osd.3 on smithi028 with /dev/vg_nvme/lv_3... 2024-09-19T09:26:00.413 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:26:00.675 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:01.103 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:00 smithi028 ceph-mon[30972]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:01.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:00 smithi078 ceph-mon[35619]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:01.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:00 smithi038 ceph-mon[35271]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:01.712 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:26:01.428+0000 7faf38891640 -1 osd.2 0 log_to_monitors true 2024-09-19T09:26:02.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:01 smithi028 ceph-mon[30972]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:01 smithi028 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.029 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:01 smithi078 ceph-mon[35619]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.029 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:01 smithi078 ceph-mon[35619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:01 smithi038 ceph-mon[35271]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:01 smithi038 ceph-mon[35271]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:26:02.804 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:02.805 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:26:02.805 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:26:02.805 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:02.805 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:02.805 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:02 smithi078 ceph-mon[35619]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:03.081 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:02 smithi028 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:03.084 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:26:03.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_3 2024-09-19T09:26:03.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:26:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:26:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: from='osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:02 smithi038 ceph-mon[35271]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:26:03.379 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 -- 172.21.15.28:0/2055038855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 msgr2=0x7fb934102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 --2- 172.21.15.28:0/2055038855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb934102150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb928009900 tx=0x7fb92802a410 comp rx=0 tx=0).stop 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 -- 172.21.15.28:0/2055038855 shutdown_connections 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 --2- 172.21.15.28:0/2055038855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 0x7fb934108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 --2- 172.21.15.28:0/2055038855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb934102150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 --2- 172.21.15.28:0/2055038855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 -- 172.21.15.28:0/2055038855 >> 172.21.15.28:0/2055038855 conn(0x7fb9340fc280 msgr2=0x7fb9340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 -- 172.21.15.28:0/2055038855 shutdown_connections 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.838+0000 7fb93c221640 1 -- 172.21.15.28:0/2055038855 wait complete. 2024-09-19T09:26:03.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 Processor -- start 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 -- start start 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 0x7fb93419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb93410c750 con 0x7fb934101cd0 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb93410c5d0 con 0x7fb934102690 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93c221640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb93410c8d0 con 0x7fb934100ad0 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb9397bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb9397bc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48436/0 (socket says 172.21.15.28:48436) 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb9397bc640 1 -- 172.21.15.28:0/2958288784 learned_addr learned my addr 172.21.15.28:0/2958288784 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb939fbd640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.839+0000 7fb93a7be640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 0x7fb93419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 -- 172.21.15.28:0/2958288784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 msgr2=0x7fb934197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 -- 172.21.15.28:0/2958288784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 msgr2=0x7fb93419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 0x7fb93419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 -- 172.21.15.28:0/2958288784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93419fb90 con 0x7fb934101cd0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb939fbd640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb9397bc640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb928009230 tx=0x7fb928031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92803f030 con 0x7fb934101cd0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93419fe20 con 0x7fb934101cd0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb928003ad0 con 0x7fb934101cd0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a0280 con 0x7fb934101cd0 2024-09-19T09:26:03.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.840+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928035070 con 0x7fb934101cd0 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.841+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918005290 con 0x7fb934101cd0 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.843+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fb928042030 con 0x7fb934101cd0 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.844+0000 7fb926ffd640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 0x7fb9200a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.844+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fb9280ee9b0 con 0x7fb934101cd0 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.844+0000 7fb939fbd640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 0x7fb9200a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.845+0000 7fb939fbd640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 0x7fb9200a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb934101b30 tx=0x7fb930005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:03.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:03.848+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9280bc730 con 0x7fb934101cd0 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi078 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-19T09:26:04.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:04.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:04.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:03 smithi028 ceph-mon[30972]: from='osd.2 ' entity='osd.2' 2024-09-19T09:26:04.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:04.034+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb918000bc0 con 0x7fb9200a31e0 2024-09-19T09:26:04.106 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:26:03.607+0000 7faf33ff1640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T09:26:04.106 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:26:03.612+0000 7faf2c623640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi078 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-19T09:26:04.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:04.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:04.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:03 smithi078 ceph-mon[35619]: from='osd.2 ' entity='osd.2' 2024-09-19T09:26:04.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi078 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:04.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:04.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:04.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:03 smithi038 ceph-mon[35271]: from='osd.2 ' entity='osd.2' 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] boot 2024-09-19T09:26:05.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:26:05.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:05.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:26:05.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] boot 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:26:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:05.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:26:05.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: osd.2 [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] boot 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:26:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:26:06.160 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:06.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:06.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]: dispatch 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]': finished 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:26:06.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:07.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]: dispatch 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]': finished 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:26:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:07.264 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:06 smithi028 sudo[49464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:07.264 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:06 smithi028 sudo[49464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:07.264 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:06 smithi028 sudo[49464]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:07.264 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:06 smithi028 sudo[49464]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:07.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]: dispatch 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1914879047' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "225d1570-ea3d-414c-93b3-f440f93bf3f3"}]': finished 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:26:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:07.763 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:07 smithi028 sudo[49516]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:07.763 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:07 smithi028 sudo[49516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:07.763 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:07 smithi028 sudo[49516]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:07.764 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:26:07 smithi028 sudo[49516]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:07.887+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb92803b460 con 0x7fb934101cd0 2024-09-19T09:26:08.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:07 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3416149980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:08.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:07 smithi038 ceph-mon[35271]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:26:08.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:08.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:08.002 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:07 smithi038 sudo[43894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:08.002 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:07 smithi038 sudo[43894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:08.002 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:07 smithi038 sudo[43894]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:08.002 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:07 smithi038 sudo[43894]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:08.002 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:08 smithi038 sudo[43897]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:08.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:07 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3416149980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:08.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:07 smithi028 ceph-mon[30972]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:26:08.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:08.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:08.271 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:07 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3416149980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:08.271 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:07 smithi078 ceph-mon[35619]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:26:08.271 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:08.271 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:08.425 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:08 smithi038 sudo[43897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:08.425 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:08 smithi038 sudo[43897]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:08.426 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:26:08 smithi038 sudo[43897]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:08.528 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:08.528 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:08.528 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44588]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:08.528 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44588]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:08.856 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:08.856 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:08.857 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44591]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:08.857 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:26:08 smithi078 sudo[44591]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:09.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[49898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[49898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[49898]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[49898]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:09.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 ceph-mon[35619]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:09.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 ceph-mon[35619]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:26:09.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.719 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:09.719 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44594]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:09.719 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44594]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:09.719 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44594]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:09.719 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44594]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[50091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[50091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[50091]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 sudo[50091]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 ceph-mon[30972]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 ceph-mon[30972]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:09 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:09.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:09 smithi038 ceph-mon[35271]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:09.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:09 smithi038 ceph-mon[35271]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:26:09.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:09.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:09 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:10.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:09 smithi078 sudo[44597]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:10.477 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43900]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:26:10.477 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43900]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:10.477 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43900]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:10.477 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43900]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43903]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43903]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43903]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 sudo[43903]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:26:10.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:26:11.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:11 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:11.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:11 smithi028 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:11.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:11 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:11.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:11 smithi078 ceph-mon[35619]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:11.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:11 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:26:11.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:11 smithi038 ceph-mon[35271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:26:12.749 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:12 smithi028 ceph-mon[30972]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:12.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:12 smithi078 ceph-mon[35619]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:12.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:12 smithi038 ceph-mon[35271]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:14.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:14 smithi028 ceph-mon[30972]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:14.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:14 smithi078 ceph-mon[35619]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:14.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:14 smithi038 ceph-mon[35271]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:26:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:15 smithi028 ceph-mon[30972]: Deploying daemon osd.3 on smithi028 2024-09-19T09:26:15.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:26:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:15 smithi078 ceph-mon[35619]: Deploying daemon osd.3 on smithi028 2024-09-19T09:26:15.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:26:15.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:15.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:15 smithi038 ceph-mon[35271]: Deploying daemon osd.3 on smithi028 2024-09-19T09:26:16.686 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:16 smithi028 ceph-mon[30972]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:26:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:16 smithi078 ceph-mon[35619]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:26:16.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:16 smithi038 ceph-mon[35271]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:26:18.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:18 smithi028 ceph-mon[30972]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:26:18.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:18 smithi078 ceph-mon[35619]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:26:18.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:18 smithi038 ceph-mon[35271]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:26:20.681 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:20 smithi028 ceph-mon[30972]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:26:20.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:20 smithi078 ceph-mon[35619]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:26:20.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:20 smithi038 ceph-mon[35271]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:26:21.700 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:21.700 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:21.700 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:21.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:21.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:21.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:21.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:21.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:21.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:22 smithi028 ceph-mon[30972]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:23.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:22 smithi078 ceph-mon[35619]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:23.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:22 smithi038 ceph-mon[35271]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.549+0000 7fb926ffd640 1 -- 172.21.15.28:0/2958288784 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb918000bc0 con 0x7fb9200a31e0 2024-09-19T09:26:24.551 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 3 on host 'smithi028' 2024-09-19T09:26:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 msgr2=0x7fb9200a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 0x7fb9200a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb934101b30 tx=0x7fb930005f90 comp rx=0 tx=0).stop 2024-09-19T09:26:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 msgr2=0x7fb9341984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb928009230 tx=0x7fb928031b60 comp rx=0 tx=0).stop 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 shutdown_connections 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9200a31e0 0x7fb9200a56a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb934102690 0x7fb93419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb934101cd0 0x7fb9341984a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 --2- 172.21.15.28:0/2958288784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934100ad0 0x7fb934197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.552+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 >> 172.21.15.28:0/2958288784 conn(0x7fb9340fc280 msgr2=0x7fb93406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.553+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 shutdown_connections 2024-09-19T09:26:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:26:24.553+0000 7fb93c221640 1 -- 172.21.15.28:0/2958288784 wait complete. 2024-09-19T09:26:24.859 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:24.860 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:25.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:25.211 DEBUG:teuthology.orchestra.run.smithi028:osd.3> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T09:26:25.215 INFO:tasks.cephadm:Deploying osd.4 on smithi038 with /dev/vg_nvme/lv_3... 2024-09-19T09:26:25.215 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:26:25.474 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:26:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:26 smithi038 ceph-mon[35271]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:27.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:26 smithi028 ceph-mon[30972]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:27.014 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:26:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:26:26.748+0000 7f66634d3640 -1 osd.3 0 log_to_monitors true 2024-09-19T09:26:27.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:26 smithi078 ceph-mon[35619]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:26:27.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:27 smithi038 ceph-mon[35271]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:26:27.984 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:27 smithi028 ceph-mon[30972]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:26:28.007 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:27 smithi078 ceph-mon[35619]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:26:28.037 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:26:28.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-09-19T09:26:28.324 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- 172.21.15.38:0/2629294766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 msgr2=0x7fe33c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 --2- 172.21.15.38:0/2629294766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c102150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- 172.21.15.38:0/2629294766 shutdown_connections 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 --2- 172.21.15.38:0/2629294766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c102150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 --2- 172.21.15.38:0/2629294766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c102690 0x7fe33c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 --2- 172.21.15.38:0/2629294766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c100ad0 0x7fe33c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- 172.21.15.38:0/2629294766 >> 172.21.15.38:0/2629294766 conn(0x7fe33c0fc280 msgr2=0x7fe33c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- 172.21.15.38:0/2629294766 shutdown_connections 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- 172.21.15.38:0/2629294766 wait complete. 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 Processor -- start 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.789+0000 7fe343bdc640 1 -- start start 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c100ad0 0x7fe33c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c102690 0x7fe33c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c10c780 con 0x7fe33c100ad0 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c10c600 con 0x7fe33c101cd0 2024-09-19T09:26:28.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c10c900 con 0x7fe33c102690 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37014/0 (socket says 172.21.15.38:37014) 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 -- 172.21.15.38:0/3167677251 learned_addr learned my addr 172.21.15.38:0/3167677251 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341978640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c100ad0 0x7fe33c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe342179640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c102690 0x7fe33c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 -- 172.21.15.38:0/3167677251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c102690 msgr2=0x7fe33c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c102690 0x7fe33c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 -- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c100ad0 msgr2=0x7fe33c19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c100ad0 0x7fe33c19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 -- 172.21.15.38:0/3167677251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c1a1dc0 con 0x7fe33c101cd0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe341177640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe330032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33003f030 con 0x7fe33c101cd0 2024-09-19T09:26:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1a2050 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe330003ce0 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.790+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035070 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.791+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a2580 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.792+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c1a27e0 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.793+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe330042030 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.793+0000 7fe33affd640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 0x7fe3240a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.793+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7fe3300eece0 con 0x7fe33c101cd0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.794+0000 7fe341978640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 0x7fe3240a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.794+0000 7fe341978640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 0x7fe3240a5730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0099a0 tx=0x7fe32c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.797+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3300bc6f0 con 0x7fe33c101cd0 2024-09-19T09:26:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:28.981+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe33c06a710 con 0x7fe3240a3270 2024-09-19T09:26:28.982 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi028 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi028 to 8546M 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:28.983 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:28.984 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:28.984 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.014 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:26:28.738+0000 7f665ec33640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T09:26:29.014 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:26:28.744+0000 7f665b26d640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:26:29.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:29.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi028 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:29.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:29.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi028 to 8546M 2024-09-19T09:26:29.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:29.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:29.017 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi028 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi028 to 8546M 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:30.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:30.096 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:26:30.096 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:26:30.096 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.096 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.096 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:30.097 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:30.097 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:30.097 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:30.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:30.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:30.745 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] boot 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:30 smithi028 ceph-mon[30972]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] boot 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.031 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:30 smithi038 ceph-mon[35271]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: osd.3 [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] boot 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:26:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:30 smithi078 ceph-mon[35619]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:26:32.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/2805251851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]': finished 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:32.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:32 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/1420599942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T09:26:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/2805251851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]': finished 2024-09-19T09:26:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:26:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:32 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/1420599942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/2805251851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]: dispatch 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c9ef00f-d335-4c90-8ad3-541a898ddb60"}]': finished 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:32 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/1420599942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:33.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:33 smithi028 ceph-mon[30972]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:33.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:33 smithi028 ceph-mon[30972]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:26:33.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:33 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:33 smithi078 ceph-mon[35619]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:33 smithi078 ceph-mon[35619]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:26:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:33 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:33 smithi038 ceph-mon[35271]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:33 smithi038 ceph-mon[35271]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:26:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:33 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:34.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:34 smithi078 ceph-mon[35619]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:34.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:34 smithi078 ceph-mon[35619]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:26:34.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:34 smithi038 ceph-mon[35271]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:34 smithi038 ceph-mon[35271]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:26:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:34 smithi028 ceph-mon[30972]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:34 smithi028 ceph-mon[30972]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:26:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:36.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:36 smithi078 ceph-mon[35619]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:26:36.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:36 smithi038 ceph-mon[35271]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:26:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:36 smithi028 ceph-mon[30972]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:26:38.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:38 smithi078 ceph-mon[35619]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:26:38.871 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:38 smithi038 ceph-mon[35271]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:26:39.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:38 smithi028 ceph-mon[30972]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:26:40.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:40 smithi038 ceph-mon[35271]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T09:26:40.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:40 smithi078 ceph-mon[35619]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T09:26:41.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:40 smithi028 ceph-mon[30972]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-19T09:26:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:26:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:41 smithi038 ceph-mon[35271]: Deploying daemon osd.4 on smithi038 2024-09-19T09:26:41.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:26:41.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:41.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:41 smithi078 ceph-mon[35619]: Deploying daemon osd.4 on smithi038 2024-09-19T09:26:42.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:26:42.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:42.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:41 smithi028 ceph-mon[30972]: Deploying daemon osd.4 on smithi038 2024-09-19T09:26:42.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:42 smithi078 ceph-mon[35619]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T09:26:42.870 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:42 smithi038 ceph-mon[35271]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T09:26:43.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:42 smithi028 ceph-mon[30972]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T09:26:44.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:44 smithi078 ceph-mon[35619]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:26:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:44 smithi038 ceph-mon[35271]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:26:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:44 smithi028 ceph-mon[30972]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:26:46.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:46 smithi078 ceph-mon[35619]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:26:46.862 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:46 smithi038 ceph-mon[35271]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:26:47.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:46 smithi028 ceph-mon[30972]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:26:47.855 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:47.855 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:47.855 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:47.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:47.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:47.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:48.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:49 smithi038 ceph-mon[35271]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:49 smithi028 ceph-mon[30972]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:49.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:49 smithi078 ceph-mon[35619]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.267+0000 7fe33affd640 1 -- 172.21.15.38:0/3167677251 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe33c06a710 con 0x7fe3240a3270 2024-09-19T09:26:50.269 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 4 on host 'smithi038' 2024-09-19T09:26:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 msgr2=0x7fe3240a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 0x7fe3240a5730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0099a0 tx=0x7fe32c009500 comp rx=0 tx=0).stop 2024-09-19T09:26:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 msgr2=0x7fe33c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe330032ce0 comp rx=0 tx=0).stop 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 shutdown_connections 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3240a3270 0x7fe3240a5730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe33c101cd0 0x7fe33c19a6d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe33c102690 0x7fe33c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 --2- 172.21.15.38:0/3167677251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe33c100ad0 0x7fe33c19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 >> 172.21.15.38:0/3167677251 conn(0x7fe33c0fc280 msgr2=0x7fe33c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 shutdown_connections 2024-09-19T09:26:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:26:50.270+0000 7fe343bdc640 1 -- 172.21.15.38:0/3167677251 wait complete. 2024-09-19T09:26:50.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:50.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:50.906 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:50.907 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:50.932 DEBUG:teuthology.orchestra.run.smithi038:osd.4> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T09:26:50.934 INFO:tasks.cephadm:Deploying osd.5 on smithi078 with /dev/vg_nvme/lv_3... 2024-09-19T09:26:50.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:51.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:51.188 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:26:52.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:52 smithi078 ceph-mon[35619]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:53.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:52 smithi038 ceph-mon[35271]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:53.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:52 smithi028 ceph-mon[30972]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:53.630 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:26:53.405+0000 7fb4a3e85640 -1 osd.4 0 log_to_monitors true 2024-09-19T09:26:53.675 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:26:53.711 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-09-19T09:26:53.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:53.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:53.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:53.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:53.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:53.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:53.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:53.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:53.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:53.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:53.964 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.085 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:54.086 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:26:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 -- 172.21.15.78:0/1926853997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0102e00 msgr2=0x7f91e0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 --2- 172.21.15.78:0/1926853997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0102e00 0x7f91e0103280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009900 tx=0x7f91cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 -- 172.21.15.78:0/1926853997 shutdown_connections 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 --2- 172.21.15.78:0/1926853997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91e01037c0 0x7f91e010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 --2- 172.21.15.78:0/1926853997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0102e00 0x7f91e0103280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 --2- 172.21.15.78:0/1926853997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91e0101c00 0x7f91e0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 -- 172.21.15.78:0/1926853997 >> 172.21.15.78:0/1926853997 conn(0x7f91e00fd390 msgr2=0x7f91e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 -- 172.21.15.78:0/1926853997 shutdown_connections 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.405+0000 7f91e6cae640 1 -- 172.21.15.78:0/1926853997 wait complete. 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 Processor -- start 2024-09-19T09:26:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 -- start start 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91e0102e00 0x7f91e01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91e01037c0 0x7f91e019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91e010d8e0 con 0x7f91e0102e00 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91e010d760 con 0x7f91e01037c0 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e6cae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91e010da60 con 0x7f91e0101c00 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36636/0 (socket says 172.21.15.78:36636) 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 -- 172.21.15.78:0/3682423247 learned_addr learned my addr 172.21.15.78:0/3682423247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 -- 172.21.15.78:0/3682423247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91e01037c0 msgr2=0x7f91e019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91e01037c0 0x7f91e019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 -- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91e0102e00 msgr2=0x7f91e01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91e0102e00 0x7f91e01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 -- 172.21.15.78:0/3682423247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e019fb90 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91e4a4a640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f91d400d770 tx=0x7f91d400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.406+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4017070 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.407+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d4009c30 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.407+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e019fe80 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.407+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4003da0 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.407+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a03b0 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.408+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0109050 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.409+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f91d4003f40 con 0x7f91e0101c00 2024-09-19T09:26:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.410+0000 7f91ddffb640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 0x7f91d00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.410+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3805+0+0 (secure 0 0 0) 0x7f91d40ce190 con 0x7f91e0101c00 2024-09-19T09:26:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.410+0000 7f91dffff640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 0x7f91d00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.411+0000 7f91dffff640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 0x7f91d00a5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0096c0 tx=0x7f91cc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.414+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f91d409c050 con 0x7f91e0101c00 2024-09-19T09:26:54.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:26:54.590+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f91e0062630 con 0x7f91d00a3270 2024-09-19T09:26:55.003 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:55.003 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi038 2024-09-19T09:26:55.003 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T09:26:55.003 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:55.004 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:54 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi038 2024-09-19T09:26:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T09:26:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:26:55.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:26:55.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:55.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:55.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:55.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:54 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:55.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:26:54.741+0000 7fb49f5e5640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T09:26:55.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:26:54.748+0000 7fb49bc1f640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi038 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:26:55.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:26:55.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:26:55.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:54 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:55.965 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:55 smithi078 ceph-mon[35619]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:55.965 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:55 smithi078 ceph-mon[35619]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:26:55.965 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:55 smithi078 ceph-mon[35619]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T09:26:55.966 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:55.966 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:55 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:56.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:55 smithi028 ceph-mon[30972]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:56.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:55 smithi028 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:26:56.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:55 smithi028 ceph-mon[30972]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T09:26:56.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:56.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:55 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:56.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:55 smithi038 ceph-mon[35271]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:56.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:55 smithi038 ceph-mon[35271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:26:56.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:55 smithi038 ceph-mon[35271]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T09:26:56.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:56.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:55 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:57.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:26:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:26:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] boot 2024-09-19T09:26:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:26:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:56 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:57.076 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:26:57.076 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:26:57.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:57.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] boot 2024-09-19T09:26:57.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:26:57.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:56 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:57.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:26:57.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:26:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:26:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: osd.4 [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] boot 2024-09-19T09:26:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:26:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:56 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:26:57.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/2192937641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]': finished 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:26:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:57 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/2335462505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/2192937641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]': finished 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:26:58.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:57 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/2335462505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:58.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T09:26:58.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/2192937641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:58.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]: dispatch 2024-09-19T09:26:58.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05e37b87-069a-444c-9428-a4f59d08116d"}]': finished 2024-09-19T09:26:58.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:26:58.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:26:58.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:57 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/2335462505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:26:59.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:26:58 smithi028 ceph-mon[30972]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:26:59.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:26:58 smithi078 ceph-mon[35619]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:26:59.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:26:58 smithi038 ceph-mon[35271]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:01.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:00 smithi028 ceph-mon[30972]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:01.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:00 smithi078 ceph-mon[35619]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:01.429 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:00 smithi038 ceph-mon[35271]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:03.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:02 smithi028 ceph-mon[30972]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:03.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:02 smithi078 ceph-mon[35619]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:03.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:02 smithi038 ceph-mon[35271]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:05.035 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:04 smithi078 ceph-mon[35619]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:05.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:04 smithi028 ceph-mon[30972]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:05.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:04 smithi038 ceph-mon[35271]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:27:06.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:06.324 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:27:06.324 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:06.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:27:06.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:07.161 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:06 smithi078 ceph-mon[35619]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:07.161 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:06 smithi078 ceph-mon[35619]: Deploying daemon osd.5 on smithi078 2024-09-19T09:27:07.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:06 smithi028 ceph-mon[30972]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:07.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:06 smithi028 ceph-mon[30972]: Deploying daemon osd.5 on smithi078 2024-09-19T09:27:07.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:06 smithi038 ceph-mon[35271]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:07.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:06 smithi038 ceph-mon[35271]: Deploying daemon osd.5 on smithi078 2024-09-19T09:27:09.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:09 smithi078 ceph-mon[35619]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:09.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:09 smithi028 ceph-mon[30972]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:09.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:09 smithi038 ceph-mon[35271]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:11.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:11 smithi078 ceph-mon[35619]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:11.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:11 smithi028 ceph-mon[30972]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:11 smithi038 ceph-mon[35271]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:13.253 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:13 smithi078 ceph-mon[35619]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:13.254 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:13.254 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:13.254 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:13.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:13 smithi028 ceph-mon[30972]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:13.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:13.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:13.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:13.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:13 smithi038 ceph-mon[35271]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:15.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:15.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:15.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:15.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.765+0000 7f91ddffb640 1 -- 172.21.15.78:0/3682423247 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91e0062630 con 0x7f91d00a3270 2024-09-19T09:27:15.766 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 5 on host 'smithi078' 2024-09-19T09:27:15.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 msgr2=0x7f91d00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 0x7f91d00a5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0096c0 tx=0x7f91cc031400 comp rx=0 tx=0).stop 2024-09-19T09:27:15.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 msgr2=0x7f91e0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f91d400d770 tx=0x7f91d400dc30 comp rx=0 tx=0).stop 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 shutdown_connections 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91d00a3270 0x7f91d00a5730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91e01037c0 0x7f91e019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91e0102e00 0x7f91e01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 --2- 172.21.15.78:0/3682423247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91e0101c00 0x7f91e0197f60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 >> 172.21.15.78:0/3682423247 conn(0x7f91e00fd390 msgr2=0x7f91e0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.767+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 shutdown_connections 2024-09-19T09:27:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:27:15.768+0000 7f91e6cae640 1 -- 172.21.15.78:0/3682423247 wait complete. 2024-09-19T09:27:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:16 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:16 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:16 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:16 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:16 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:16 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.469 DEBUG:teuthology.orchestra.run.smithi078:osd.5> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T09:27:16.471 INFO:tasks.cephadm:Deploying osd.6 on smithi028 with /dev/vg_nvme/lv_2... 2024-09-19T09:27:16.472 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:27:16.500 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:16.500 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.500 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:16.736 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:17.200 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:17 smithi028 ceph-mon[30972]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:17.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:17 smithi078 ceph-mon[35619]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:17.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:17 smithi038 ceph-mon[35271]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:18.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:27:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:27:18.455+0000 7f2bfbcb3640 -1 osd.5 0 log_to_monitors true 2024-09-19T09:27:19.275 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:19 smithi078 ceph-mon[35619]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:19.275 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:19 smithi078 ceph-mon[35619]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.275 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:19 smithi078 ceph-mon[35619]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.350 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:19 smithi028 ceph-mon[30972]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:19.350 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:19 smithi028 ceph-mon[30972]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.350 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:19 smithi028 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.423 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:27:19.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:19 smithi038 ceph-mon[35271]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:19.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:19 smithi038 ceph-mon[35271]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:19 smithi038 ceph-mon[35271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:27:19.453 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_2 2024-09-19T09:27:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi078 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.249 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:20.250 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 -- 172.21.15.28:0/752809384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 msgr2=0x7f9860104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 --2- 172.21.15.28:0/752809384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860104ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9850009900 tx=0x7f985002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 -- 172.21.15.28:0/752809384 shutdown_connections 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 --2- 172.21.15.28:0/752809384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 --2- 172.21.15.28:0/752809384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f98601073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 --2- 172.21.15.28:0/752809384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860104ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 -- 172.21.15.28:0/752809384 >> 172.21.15.28:0/752809384 conn(0x7f98600fc280 msgr2=0x7f98600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 -- 172.21.15.28:0/752809384 shutdown_connections 2024-09-19T09:27:20.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 -- 172.21.15.28:0/752809384 wait complete. 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.334+0000 7f986511a640 1 Processor -- start 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 -- start start 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f9860100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f986010d860 con 0x7f98601026c0 2024-09-19T09:27:20.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f986010d6e0 con 0x7f9860107920 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f986511a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f986010d9e0 con 0x7f9860104ff0 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f985ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f985ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40978/0 (socket says 172.21.15.28:40978) 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f985ed9d640 1 -- 172.21.15.28:0/1749521987 learned_addr learned my addr 172.21.15.28:0/1749521987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f985e59c640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f9860100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.335+0000 7f985f59e640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 -- 172.21.15.28:0/1749521987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 msgr2=0x7f9860100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 -- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 msgr2=0x7f9860102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 -- 172.21.15.28:0/1749521987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860101470 con 0x7f9860104ff0 2024-09-19T09:27:20.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985ed9d640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985f59e640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f985e59c640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f9860100670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f984800d6a0 tx=0x7f984800db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848017070 con 0x7f9860104ff0 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9848009820 con 0x7f9860104ff0 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98601a20d0 con 0x7f9860104ff0 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848004d00 con 0x7f9860104ff0 2024-09-19T09:27:20.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.336+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601a25e0 con 0x7f9860104ff0 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.337+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98600fdd70 con 0x7f9860104ff0 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.338+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f98480099d0 con 0x7f9860104ff0 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.339+0000 7f9857fff640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 0x7f984c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.339+0000 7f985ed9d640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 0x7f984c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.339+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f98480ce3e0 con 0x7f9860104ff0 2024-09-19T09:27:20.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.340+0000 7f985ed9d640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 0x7f984c0a5700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9850031440 tx=0x7f98500315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.344+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f984809bfa0 con 0x7f9860104ff0 2024-09-19T09:27:20.356 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:27:20.051+0000 7f2bf7413640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T09:27:20.356 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:27:20.058+0000 7f2bf3a4d640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi078 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:20.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:20.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi078 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:20.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:20.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:20.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:20.529+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9860062630 con 0x7f984c0a3240 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] boot 2024-09-19T09:27:21.235 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:27:21.236 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.236 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:21.236 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:21.236 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:21 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:21.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] boot 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:21 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: osd.5 [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] boot 2024-09-19T09:27:21.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:27:21.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:27:21.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:21.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:21.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:21 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:22.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:22 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:27:22.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:22 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:27:22.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:22 smithi028 ceph-mon[30972]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:22.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:22 smithi028 ceph-mon[30972]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T09:27:22.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:22 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:27:22.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:22 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:27:22.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:22 smithi038 ceph-mon[35271]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:22.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:22 smithi038 ceph-mon[35271]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T09:27:22.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:22 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:27:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:22 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:27:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:22 smithi078 ceph-mon[35619]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:22 smithi078 ceph-mon[35619]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]: dispatch 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]': finished 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:27:23.380 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:23 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:23.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T09:27:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]: dispatch 2024-09-19T09:27:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]': finished 2024-09-19T09:27:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:27:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:23 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T09:27:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]: dispatch 2024-09-19T09:27:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1880743007' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8"}]': finished 2024-09-19T09:27:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:27:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:23 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:24.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:24 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/428748034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:24.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:24 smithi038 ceph-mon[35271]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:27:24.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:24.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:24 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/428748034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:24.644 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:24 smithi028 ceph-mon[30972]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:27:24.644 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:24.645 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:24 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/428748034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:24.645 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:24 smithi078 ceph-mon[35619]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:27:24.645 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:25.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:25 smithi038 ceph-mon[35271]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:25.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:25 smithi028 ceph-mon[30972]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:25.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:25 smithi078 ceph-mon[35619]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:27.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:27 smithi038 ceph-mon[35271]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:27:27.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:27 smithi028 ceph-mon[30972]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:27:27.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:27 smithi078 ceph-mon[35619]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:27:29.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:29 smithi028 ceph-mon[30972]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:27:29.409 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:29 smithi078 ceph-mon[35619]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:27:29.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:29 smithi038 ceph-mon[35271]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:27:31.410 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:31 smithi028 ceph-mon[30972]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T09:27:31.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:31 smithi038 ceph-mon[35271]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T09:27:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:31 smithi078 ceph-mon[35619]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T09:27:32.414 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:27:32.414 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:27:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:32.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:27:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:33.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:33 smithi038 ceph-mon[35271]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T09:27:33.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:33 smithi038 ceph-mon[35271]: Deploying daemon osd.6 on smithi028 2024-09-19T09:27:33.474 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:33 smithi028 ceph-mon[30972]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T09:27:33.475 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:33 smithi028 ceph-mon[30972]: Deploying daemon osd.6 on smithi028 2024-09-19T09:27:33.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:33 smithi078 ceph-mon[35619]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T09:27:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:33 smithi078 ceph-mon[35619]: Deploying daemon osd.6 on smithi028 2024-09-19T09:27:35.413 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:35 smithi028 ceph-mon[30972]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:27:35.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:35 smithi038 ceph-mon[35271]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:27:35.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:35 smithi078 ceph-mon[35619]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:27:37.359 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:37 smithi028 ceph-mon[30972]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:37.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:37 smithi038 ceph-mon[35271]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:37.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:37 smithi078 ceph-mon[35619]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:38.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:38.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:38.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:38.473 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:38.474 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:38.474 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:38.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:38.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:38.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:39.194 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:39 smithi028 ceph-mon[30972]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:39.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:39 smithi038 ceph-mon[35271]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:39.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:39 smithi078 ceph-mon[35619]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.875+0000 7f9857fff640 1 -- 172.21.15.28:0/1749521987 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9860062630 con 0x7f984c0a3240 2024-09-19T09:27:40.877 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 6 on host 'smithi028' 2024-09-19T09:27:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 msgr2=0x7f984c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 0x7f984c0a5700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9850031440 tx=0x7f98500315b0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 msgr2=0x7f9860100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f9860100670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f984800d6a0 tx=0x7f984800db60 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 shutdown_connections 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f984c0a3240 0x7f984c0a5700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9860107920 0x7f9860100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9860104ff0 0x7f9860100670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 --2- 172.21.15.28:0/1749521987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98601026c0 0x7f9860102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 >> 172.21.15.28:0/1749521987 conn(0x7f98600fc280 msgr2=0x7f9860102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 shutdown_connections 2024-09-19T09:27:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:27:40.878+0000 7f986511a640 1 -- 172.21.15.28:0/1749521987 wait complete. 2024-09-19T09:27:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:41 smithi038 ceph-mon[35271]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:41 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:41 smithi028 ceph-mon[30972]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:41 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.554 DEBUG:teuthology.orchestra.run.smithi028:osd.6> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T09:27:41.557 INFO:tasks.cephadm:Deploying osd.7 on smithi038 with /dev/vg_nvme/lv_2... 2024-09-19T09:27:41.557 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:27:41.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:41 smithi078 ceph-mon[35619]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:41.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:41 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:27:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:42 smithi028 ceph-mon[30972]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:42 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:43.043 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:42 smithi038 ceph-mon[35271]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:43.043 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:43.043 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:42 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:42 smithi078 ceph-mon[35619]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:42 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:44.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:27:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:27:43.996+0000 7f5405799640 -1 osd.6 0 log_to_monitors true 2024-09-19T09:27:44.417 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:27:44.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-09-19T09:27:44.704 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:27:45.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:44 smithi028 ceph-mon[30972]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:44 smithi028 ceph-mon[30972]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:27:45.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:44 smithi038 ceph-mon[35271]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:45.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:44 smithi038 ceph-mon[35271]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:27:45.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:44 smithi078 ceph-mon[35619]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:44 smithi078 ceph-mon[35619]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:27:45.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 -- 172.21.15.38:0/293387035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8100c80 msgr2=0x7f40c8101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 --2- 172.21.15.38:0/293387035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8100c80 0x7f40c8101100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009900 tx=0x7f40b802a410 comp rx=0 tx=0).stop 2024-09-19T09:27:45.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 -- 172.21.15.38:0/293387035 shutdown_connections 2024-09-19T09:27:45.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 --2- 172.21.15.38:0/293387035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8100c80 0x7f40c8101100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 --2- 172.21.15.38:0/293387035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8101640 0x7f40c8105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 --2- 172.21.15.38:0/293387035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 0x7f40c8100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 -- 172.21.15.38:0/293387035 >> 172.21.15.38:0/293387035 conn(0x7f40c80fba80 msgr2=0x7f40c80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 -- 172.21.15.38:0/293387035 shutdown_connections 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.231+0000 7f40cc99d640 1 -- 172.21.15.38:0/293387035 wait complete. 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 Processor -- start 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 -- start start 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 0x7f40c819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8100c80 0x7f40c819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8113e90 con 0x7f40c8100c80 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8113d10 con 0x7f40c8101640 2024-09-19T09:27:45.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40cc99d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8114010 con 0x7f40c80fff10 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36040/0 (socket says 172.21.15.38:36040) 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8100c80 0x7f40c819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 -- 172.21.15.38:0/3286822473 learned_addr learned my addr 172.21.15.38:0/3286822473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c659c640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 0x7f40c819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 -- 172.21.15.38:0/3286822473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 msgr2=0x7f40c819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 0x7f40c819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 -- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8100c80 msgr2=0x7f40c819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8100c80 0x7f40c819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.232+0000 7f40c6d9d640 1 -- 172.21.15.38:0/3286822473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c81a60a0 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40c6d9d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e970 tx=0x7f40bc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00ccc0 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c81a6390 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40bc00ce60 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc010490 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.233+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81a68a0 con 0x7f40c8101640 2024-09-19T09:27:45.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.234+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c81a6b60 con 0x7f40c8101640 2024-09-19T09:27:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.235+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f40bc005020 con 0x7f40c8101640 2024-09-19T09:27:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.236+0000 7f40b77fe640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 0x7f40b00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.236+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4669+0+0 (secure 0 0 0) 0x7f40bc0cef80 con 0x7f40c8101640 2024-09-19T09:27:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.236+0000 7f40c659c640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 0x7f40b00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.236+0000 7f40c659c640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 0x7f40b00a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40c0007350 tx=0x7f40c00072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.240+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f40bc09cae0 con 0x7f40c8101640 2024-09-19T09:27:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:27:45.424+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f40c81a6d30 con 0x7f40b00a3270 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi028 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.001 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi028 to 5697M 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:46.002 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.014 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:27:45.733+0000 7f5400ef9640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T09:27:46.014 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:27:45.747+0000 7f53fd533640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi028 2024-09-19T09:27:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi028 to 5697M 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:46.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:46.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi028 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:27:46.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi028 to 5697M 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:27:46.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:47.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:47.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:47.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:27:47.034 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T09:27:47.035 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:47.035 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: from='osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] boot 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T09:27:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.055 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:27:48.055 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:27:48.055 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.055 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] boot 2024-09-19T09:27:48.056 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T09:27:48.056 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: osd.6 [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] boot 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T09:27:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:27:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/2117107989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T09:27:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]': finished 2024-09-19T09:27:49.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:27:49.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:27:49.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/9344329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/2117107989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]': finished 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:27:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/9344329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/2117107989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]: dispatch 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19962bf-fa07-4593-96c4-8f8572d156dd"}]': finished 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:27:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/9344329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:27:51.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:50 smithi038 ceph-mon[35271]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:51.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:50 smithi028 ceph-mon[30972]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:51.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:50 smithi078 ceph-mon[35619]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:53.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:52 smithi038 ceph-mon[35271]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:53.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:52 smithi028 ceph-mon[30972]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:53.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:52 smithi078 ceph-mon[35619]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:55.165 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:54 smithi038 ceph-mon[35271]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:55.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:54 smithi028 ceph-mon[30972]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:55.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:54 smithi078 ceph-mon[35619]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:56.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:56 smithi038 ceph-mon[35271]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:57.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:56 smithi028 ceph-mon[30972]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:57.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:56 smithi078 ceph-mon[35619]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:58.037 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:27:58.037 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:58.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:27:58.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:58.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:27:58.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:59.140 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:58 smithi038 ceph-mon[35271]: Deploying daemon osd.7 on smithi038 2024-09-19T09:27:59.141 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:27:58 smithi038 ceph-mon[35271]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:59.258 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:58 smithi078 ceph-mon[35619]: Deploying daemon osd.7 on smithi038 2024-09-19T09:27:59.258 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:27:58 smithi078 ceph-mon[35619]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:27:59.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:58 smithi028 ceph-mon[30972]: Deploying daemon osd.7 on smithi038 2024-09-19T09:27:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:27:58 smithi028 ceph-mon[30972]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:00.963 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:00 smithi038 ceph-mon[35271]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:01.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:00 smithi028 ceph-mon[30972]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:01.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:00 smithi078 ceph-mon[35619]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:03.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:02 smithi038 ceph-mon[35271]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:03.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:02 smithi028 ceph-mon[30972]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:03.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:02 smithi078 ceph-mon[35619]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:04.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:04.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:04.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:04.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:04.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:04.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:04.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:05.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:05 smithi038 ceph-mon[35271]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:05 smithi028 ceph-mon[30972]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:05.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:05 smithi078 ceph-mon[35619]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.591+0000 7f40b77fe640 1 -- 172.21.15.38:0/3286822473 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f40c81a6d30 con 0x7f40b00a3270 2024-09-19T09:28:06.593 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 7 on host 'smithi038' 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 msgr2=0x7f40b00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 0x7f40b00a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40c0007350 tx=0x7f40c00072e0 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 msgr2=0x7f40c81a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e970 tx=0x7f40bc00ed70 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 shutdown_connections 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f40b00a3270 0x7f40b00a5730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40c8101640 0x7f40c81a5a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40c8100c80 0x7f40c819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 --2- 172.21.15.38:0/3286822473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c80fff10 0x7f40c819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 >> 172.21.15.38:0/3286822473 conn(0x7f40c80fba80 msgr2=0x7f40c80fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 shutdown_connections 2024-09-19T09:28:06.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:28:06.593+0000 7f40cc99d640 1 -- 172.21.15.38:0/3286822473 wait complete. 2024-09-19T09:28:06.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:07.260 DEBUG:teuthology.orchestra.run.smithi038:osd.7> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T09:28:07.262 INFO:tasks.cephadm:Deploying osd.8 on smithi078 with /dev/vg_nvme/lv_2... 2024-09-19T09:28:07.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:28:07.513 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:08.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:08 smithi028 ceph-mon[30972]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:08.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:08 smithi038 ceph-mon[35271]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:08.955 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:08 smithi078 ceph-mon[35619]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:09.824 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:09 smithi038 ceph-mon[35271]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:09.824 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:09 smithi038 ceph-mon[35271]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:09.825 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:28:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:28:09.478+0000 7fc94b956640 -1 osd.7 0 log_to_monitors true 2024-09-19T09:28:09.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:09 smithi078 ceph-mon[35619]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:09.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:09 smithi078 ceph-mon[35619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:10.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:09 smithi028 ceph-mon[30972]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:10.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:09 smithi028 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:28:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:28:10.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-09-19T09:28:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi038 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:10.959 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:10.960 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:10 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/3262009797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e341026d0 msgr2=0x7f8e34108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/3262009797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e341026d0 0x7f8e34108f60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009ca0 tx=0x7f8e2c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/3262009797 shutdown_connections 2024-09-19T09:28:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/3262009797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e341026d0 0x7f8e34108f60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/3262009797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e34101d10 0x7f8e34102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/3262009797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 0x7f8e34100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/3262009797 >> 172.21.15.78:0/3262009797 conn(0x7f8e340fc2a0 msgr2=0x7f8e340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/3262009797 shutdown_connections 2024-09-19T09:28:10.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.964+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/3262009797 wait complete. 2024-09-19T09:28:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 Processor -- start 2024-09-19T09:28:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 -- start start 2024-09-19T09:28:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 0x7f8e34197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:10.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 0x7f8e3419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410d920 con 0x7f8e341026d0 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410d7a0 con 0x7f8e34100b10 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e3a5ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410daa0 con 0x7f8e34101d10 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37172/0 (socket says 172.21.15.78:37172) 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e337fe640 1 -- 172.21.15.78:0/759954502 learned_addr learned my addr 172.21.15.78:0/759954502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.965+0000 7f8e33fff640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 0x7f8e34197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e38b6b640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 0x7f8e3419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 -- 172.21.15.78:0/759954502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 msgr2=0x7f8e34197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 0x7f8e34197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 -- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 msgr2=0x7f8e3419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:10.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 0x7f8e3419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 -- 172.21.15.78:0/759954502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3419fae0 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e38b6b640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 0x7f8e3419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e337fe640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00c920 tx=0x7f8e1c00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c007bb0 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e1c007d50 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3419fdd0 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0187d0 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.966+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a0300 con 0x7f8e34101d10 2024-09-19T09:28:10.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.967+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e34107e40 con 0x7f8e34101d10 2024-09-19T09:28:10.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.968+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8e1c00f450 con 0x7f8e34101d10 2024-09-19T09:28:10.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.969+0000 7f8e317fa640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 0x7f8e200a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:10.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.969+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f8e1c0ced90 con 0x7f8e34101d10 2024-09-19T09:28:10.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.969+0000 7f8e33fff640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 0x7f8e200a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:10.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.970+0000 7f8e33fff640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 0x7f8e200a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e34101b70 tx=0x7f8e24009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:10.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:10.973+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8e1c014030 con 0x7f8e34101d10 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi038 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:11.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:10 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:11.147+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8e34062630 con 0x7f8e200a3270 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi038 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:11.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:10 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:11.828 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:28:11.828 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='osd.7 ' entity='osd.7' 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] boot 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T09:28:11.829 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.925 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:28:11.574+0000 7fc9480ed640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T09:28:11.925 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:28:11.587+0000 7fc93f6e8640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='osd.7 ' entity='osd.7' 2024-09-19T09:28:11.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] boot 2024-09-19T09:28:11.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T09:28:11.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:12.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='osd.7 ' entity='osd.7' 2024-09-19T09:28:12.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: osd.7 [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] boot 2024-09-19T09:28:12.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T09:28:12.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:28:12.952 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:12 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:28:12.953 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:12 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:28:12.953 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:12 smithi078 ceph-mon[35619]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:12.953 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:12 smithi078 ceph-mon[35619]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T09:28:13.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:12 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:28:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:12 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:28:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:12 smithi028 ceph-mon[30972]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:12 smithi028 ceph-mon[30972]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T09:28:13.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:12 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:28:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:12 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:28:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:12 smithi038 ceph-mon[35271]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:28:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:12 smithi038 ceph-mon[35271]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T09:28:14.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/3413324116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:13 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:13 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]': finished 2024-09-19T09:28:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:13 smithi028 ceph-mon[30972]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:28:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:14.105 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/3413324116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.105 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:13 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.105 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:13 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]': finished 2024-09-19T09:28:14.105 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:13 smithi078 ceph-mon[35619]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:28:14.105 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:14.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/3413324116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:13 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]: dispatch 2024-09-19T09:28:14.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:13 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313b7ee6-931b-4eae-afa0-1f0950c1a0db"}]': finished 2024-09-19T09:28:14.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:13 smithi038 ceph-mon[35271]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:28:14.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:15.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:14 smithi028 ceph-mon[30972]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:15.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/3322801647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:15.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:14 smithi028 ceph-mon[30972]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:28:15.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:14 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:15.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:14 smithi078 ceph-mon[35619]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:15.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/3322801647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:15.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:14 smithi078 ceph-mon[35619]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:28:15.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:14 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:15.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:14 smithi038 ceph-mon[35271]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/3322801647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:14 smithi038 ceph-mon[35271]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:28:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:14 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:16.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:16 smithi078 ceph-mon[35619]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:16 smithi078 ceph-mon[35619]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:28:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:16 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:16 smithi038 ceph-mon[35271]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:16 smithi038 ceph-mon[35271]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:28:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:16 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:17.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:16 smithi028 ceph-mon[30972]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:16 smithi028 ceph-mon[30972]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:28:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:18.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:18 smithi078 ceph-mon[35619]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:18.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:18 smithi038 ceph-mon[35271]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:19.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:18 smithi028 ceph-mon[30972]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:20.831 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:20 smithi078 ceph-mon[35619]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:20.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:20 smithi038 ceph-mon[35271]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:21.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:20 smithi028 ceph-mon[30972]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:22.842 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:22 smithi078 ceph-mon[35619]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:22.842 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:28:22.842 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:22.842 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:22 smithi078 ceph-mon[35619]: Deploying daemon osd.8 on smithi078 2024-09-19T09:28:22.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:22 smithi038 ceph-mon[35271]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:22.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:28:22.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:22.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:22 smithi038 ceph-mon[35271]: Deploying daemon osd.8 on smithi078 2024-09-19T09:28:23.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:22 smithi028 ceph-mon[30972]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:23.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:28:23.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:23.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:22 smithi028 ceph-mon[30972]: Deploying daemon osd.8 on smithi078 2024-09-19T09:28:24.652 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:24 smithi078 ceph-mon[35619]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:24.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:24 smithi038 ceph-mon[35271]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:25.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:24 smithi028 ceph-mon[30972]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:26.802 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:26 smithi078 ceph-mon[35619]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:26 smithi038 ceph-mon[35271]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:27.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:26 smithi028 ceph-mon[30972]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:28.717 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:28 smithi078 ceph-mon[35619]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:28.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:28.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:28.718 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:28.816 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:28 smithi028 ceph-mon[30972]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:28.816 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:28.816 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:28.816 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:28 smithi038 ceph-mon[35271]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:28.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:28.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:30.672 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:30 smithi078 ceph-mon[35619]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:30 smithi038 ceph-mon[35271]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:31.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:30 smithi028 ceph-mon[30972]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:31.332 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 8 on host 'smithi078' 2024-09-19T09:28:31.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.330+0000 7f8e317fa640 1 -- 172.21.15.78:0/759954502 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e34062630 con 0x7f8e200a3270 2024-09-19T09:28:31.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 msgr2=0x7f8e200a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 0x7f8e200a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e34101b70 tx=0x7f8e24009110 comp rx=0 tx=0).stop 2024-09-19T09:28:31.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 msgr2=0x7f8e341983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00c920 tx=0x7f8e1c00cd20 comp rx=0 tx=0).stop 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 shutdown_connections 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8e200a3270 0x7f8e200a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e341026d0 0x7f8e3419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e34101d10 0x7f8e341983f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 --2- 172.21.15.78:0/759954502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e34100b10 0x7f8e34197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 >> 172.21.15.78:0/759954502 conn(0x7f8e340fc2a0 msgr2=0x7f8e34105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 shutdown_connections 2024-09-19T09:28:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:28:31.333+0000 7f8e3a5ce640 1 -- 172.21.15.78:0/759954502 wait complete. 2024-09-19T09:28:31.949 DEBUG:teuthology.orchestra.run.smithi078:osd.8> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T09:28:31.952 INFO:tasks.cephadm:Deploying osd.9 on smithi028 with /dev/vg_nvme/lv_1... 2024-09-19T09:28:31.952 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:28:32.222 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:32.604 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.604 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.604 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:32 smithi078 ceph-mon[35619]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:32.604 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.604 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.624 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.624 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.624 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:32 smithi028 ceph-mon[30972]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:32.624 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.625 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:32 smithi038 ceph-mon[35271]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:32.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:32.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:34.357 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:28:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:28:34.019+0000 7f79e9590640 -1 osd.8 0 log_to_monitors true 2024-09-19T09:28:34.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:34 smithi038 ceph-mon[35271]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:34.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:34 smithi038 ceph-mon[35271]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:34 smithi038 ceph-mon[35271]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:34.719 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:34 smithi028 ceph-mon[30972]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:34.719 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:34 smithi028 ceph-mon[30972]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:34.719 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:34 smithi028 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:34.834 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:34 smithi078 ceph-mon[35619]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:34.834 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:34 smithi078 ceph-mon[35619]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:34.835 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:34 smithi078 ceph-mon[35619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:28:35.177 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:28:35.214 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi028:vg_nvme/lv_1 2024-09-19T09:28:35.482 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:35.683 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:28:35.683 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T09:28:35.683 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:35.683 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi078 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:35.685 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.685 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.685 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi078 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 -- 172.21.15.28:0/1186024086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 msgr2=0x7fa014108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 --2- 172.21.15.28:0/1186024086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014108f60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009ca0 tx=0x7fa00c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 -- 172.21.15.28:0/1186024086 shutdown_connections 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 --2- 172.21.15.28:0/1186024086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014101d10 0x7fa014102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 --2- 172.21.15.28:0/1186024086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014108f60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 --2- 172.21.15.28:0/1186024086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014100b10 0x7fa014100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 -- 172.21.15.28:0/1186024086 >> 172.21.15.28:0/1186024086 conn(0x7fa0140fc2a0 msgr2=0x7fa0140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 -- 172.21.15.28:0/1186024086 shutdown_connections 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.876+0000 7fa018c9f640 1 -- 172.21.15.28:0/1186024086 wait complete. 2024-09-19T09:28:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 Processor -- start 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 -- start start 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014100b10 0x7fa014104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 0x7fa0141030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa01410c770 con 0x7fa0141026d0 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa01410c5f0 con 0x7fa014100b10 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.877+0000 7fa018c9f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa01410c8f0 con 0x7fa014101d10 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa011d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 0x7fa0141030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39168/0 (socket says 172.21.15.28:39168) 2024-09-19T09:28:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 -- 172.21.15.28:0/2837773492 learned_addr learned my addr 172.21.15.28:0/2837773492 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa01259c640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014100b10 0x7fa014104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 -- 172.21.15.28:0/2837773492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 msgr2=0x7fa0141030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 0x7fa0141030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 -- 172.21.15.28:0/2837773492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014100b10 msgr2=0x7fa014104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014100b10 0x7fa014104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 -- 172.21.15.28:0/2837773492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa014103ef0 con 0x7fa0141026d0 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa011d9b640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 0x7fa0141030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa012d9d640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa00c031ae0 tx=0x7fa00c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c003b70 con 0x7fa0141026d0 2024-09-19T09:28:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa00c003d10 con 0x7fa0141026d0 2024-09-19T09:28:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0141a1f20 con 0x7fa0141026d0 2024-09-19T09:28:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c02fc20 con 0x7fa0141026d0 2024-09-19T09:28:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.878+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141a23d0 con 0x7fa0141026d0 2024-09-19T09:28:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.879+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0141a2600 con 0x7fa0141026d0 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.880+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa00c0046c0 con 0x7fa0141026d0 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.881+0000 7fa0037fe640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.881+0000 7fa01259c640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.881+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7fa00c0f0240 con 0x7fa0141026d0 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.882+0000 7fa01259c640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa014101b70 tx=0x7fa004005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:35.885+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa00c0bd990 con 0x7fa0141026d0 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi078 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:35.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:36.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:36.069+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa0141a27d0 con 0x7f9ffc0a3240 2024-09-19T09:28:36.606 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:28:36.265+0000 7f79e4cf0640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T09:28:36.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:28:36.279+0000 7f79dd322640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:36.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:36.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:36 smithi078 ceph-mon[35619]: from='osd.8 ' entity='osd.8' 2024-09-19T09:28:36.761 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:36.761 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:28:36.761 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T09:28:36.761 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:36.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:36 smithi028 ceph-mon[30972]: from='osd.8 ' entity='osd.8' 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi028:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:36 smithi038 ceph-mon[35271]: from='osd.8 ' entity='osd.8' 2024-09-19T09:28:37.686 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:28:37.686 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:28:37.686 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:37.686 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] boot 2024-09-19T09:28:37.687 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:28:37.687 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] boot 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:28:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:37.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:28:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:28:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: osd.8 [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] boot 2024-09-19T09:28:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:28:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:28:38.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:38 smithi028 ceph-mon[30972]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:38.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:38 smithi028 ceph-mon[30972]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T09:28:38.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:38 smithi078 ceph-mon[35619]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:38 smithi078 ceph-mon[35619]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T09:28:38.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:38 smithi038 ceph-mon[35271]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:28:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:38 smithi038 ceph-mon[35271]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T09:28:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T09:28:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]: dispatch 2024-09-19T09:28:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]': finished 2024-09-19T09:28:39.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:28:39.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:39.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:39 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2687867284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:39.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T09:28:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]: dispatch 2024-09-19T09:28:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]': finished 2024-09-19T09:28:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:28:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:39 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2687867284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:39.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T09:28:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]: dispatch 2024-09-19T09:28:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4135413596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6ecdd64-1d04-4526-bf82-921f29aecf1f"}]': finished 2024-09-19T09:28:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:28:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:39 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2687867284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:28:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:40 smithi028 ceph-mon[30972]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:40 smithi028 ceph-mon[30972]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:28:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:40 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:41.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:40 smithi078 ceph-mon[35619]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:41.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:40 smithi078 ceph-mon[35619]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:28:41.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:40 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:41.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:40 smithi038 ceph-mon[35271]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:41.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:40 smithi038 ceph-mon[35271]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:28:41.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:40 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:28:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:42 smithi028 ceph-mon[30972]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:43.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:42 smithi078 ceph-mon[35619]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:43.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:42 smithi038 ceph-mon[35271]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:44.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:44 smithi028 ceph-mon[30972]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-19T09:28:45.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:44 smithi078 ceph-mon[35619]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-19T09:28:45.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:44 smithi038 ceph-mon[35271]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-19T09:28:46.970 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:46 smithi028 ceph-mon[30972]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T09:28:47.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:46 smithi078 ceph-mon[35619]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T09:28:47.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:46 smithi038 ceph-mon[35271]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T09:28:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:28:48.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:28:48.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:48.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:28:48.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:48 smithi028 ceph-mon[30972]: Deploying daemon osd.9 on smithi028 2024-09-19T09:28:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:48 smithi028 ceph-mon[30972]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T09:28:49.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:48 smithi078 ceph-mon[35619]: Deploying daemon osd.9 on smithi028 2024-09-19T09:28:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:48 smithi078 ceph-mon[35619]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T09:28:49.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:48 smithi038 ceph-mon[35271]: Deploying daemon osd.9 on smithi028 2024-09-19T09:28:49.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:48 smithi038 ceph-mon[35271]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T09:28:50.950 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:50 smithi028 ceph-mon[30972]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:28:51.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:50 smithi078 ceph-mon[35619]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:28:51.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:50 smithi038 ceph-mon[35271]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T09:28:53.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:52 smithi028 ceph-mon[30972]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:28:53.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:52 smithi078 ceph-mon[35619]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:28:53.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:52 smithi038 ceph-mon[35271]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T09:28:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:54.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:54.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:54.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:54 smithi028 ceph-mon[30972]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:28:55.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:54 smithi078 ceph-mon[35619]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:28:55.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:54 smithi038 ceph-mon[35271]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:28:57.047 INFO:teuthology.orchestra.run.smithi028.stdout:Created osd(s) 9 on host 'smithi028' 2024-09-19T09:28:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.046+0000 7fa0037fe640 1 -- 172.21.15.28:0/2837773492 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa0141a27d0 con 0x7f9ffc0a3240 2024-09-19T09:28:57.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 msgr2=0x7f9ffc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:57.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa014101b70 tx=0x7fa004005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 msgr2=0x7fa014103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa00c031ae0 tx=0x7fa00c031ee0 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 shutdown_connections 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ffc0a3240 0x7f9ffc0a5700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0141026d0 0x7fa014103630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa014101d10 0x7fa0141030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 --2- 172.21.15.28:0/2837773492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014100b10 0x7fa014104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.048+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 >> 172.21.15.28:0/2837773492 conn(0x7fa0140fc2a0 msgr2=0x7fa0140fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.049+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 shutdown_connections 2024-09-19T09:28:57.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:28:57.049+0000 7fa018c9f640 1 -- 172.21.15.28:0/2837773492 wait complete. 2024-09-19T09:28:57.254 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.255 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:57.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:57 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:57 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.663 DEBUG:teuthology.orchestra.run.smithi028:osd.9> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T09:28:57.667 INFO:tasks.cephadm:Deploying osd.10 on smithi038 with /dev/vg_nvme/lv_1... 2024-09-19T09:28:57.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:28:57.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:57.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:57 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:28:57.930 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:28:59.416 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:28:59 smithi038 ceph-mon[35271]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:59.484 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:28:59 smithi028 ceph-mon[30972]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:59.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:28:59 smithi078 ceph-mon[35619]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:28:59.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:28:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:28:59.482+0000 7f803e71b640 -1 osd.9 0 log_to_monitors true 2024-09-19T09:29:00.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:00 smithi028 ceph-mon[30972]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:29:00.570 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:00 smithi038 ceph-mon[35271]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:29:00.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:00 smithi078 ceph-mon[35619]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:29:00.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T09:29:00.642 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-09-19T09:29:00.940 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.371 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.372 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.372 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.372 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:01.372 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:01.372 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:01 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.384+0000 7f6bb4401640 1 -- 172.21.15.38:0/3137593081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 msgr2=0x7f6bac108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.384+0000 7f6bb4401640 1 --2- 172.21.15.38:0/3137593081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac108f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009ca0 tx=0x7f6ba402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 -- 172.21.15.38:0/3137593081 shutdown_connections 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 --2- 172.21.15.38:0/3137593081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac108f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 --2- 172.21.15.38:0/3137593081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac101cd0 0x7f6bac102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 --2- 172.21.15.38:0/3137593081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac100ad0 0x7f6bac100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 -- 172.21.15.38:0/3137593081 >> 172.21.15.38:0/3137593081 conn(0x7f6bac0fc280 msgr2=0x7f6bac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 -- 172.21.15.38:0/3137593081 shutdown_connections 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 -- 172.21.15.38:0/3137593081 wait complete. 2024-09-19T09:29:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 Processor -- start 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 -- start start 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.385+0000 7f6bb4401640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 0x7f6bac06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac101cd0 0x7f6bac19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10c750 con 0x7f6bac100ad0 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10c5d0 con 0x7f6bac102690 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10c8d0 con 0x7f6bac101cd0 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb299e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 0x7f6bac06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb299e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60892/0 (socket says 172.21.15.38:60892) 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb299e640 1 -- 172.21.15.38:0/1281349241 learned_addr learned my addr 172.21.15.38:0/1281349241 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T09:29:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb199c640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac101cd0 0x7f6bac19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 -- 172.21.15.38:0/1281349241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac101cd0 msgr2=0x7f6bac19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac101cd0 0x7f6bac19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 -- 172.21.15.38:0/1281349241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 msgr2=0x7f6bac19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 -- 172.21.15.38:0/1281349241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac19fb30 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb299e640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb219d640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 0x7f6bac06aad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9c0 tx=0x7f6ba800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ccc0 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac19fdc0 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba80035f0 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.386+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ce60 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.387+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1a0220 con 0x7f6bac100ad0 2024-09-19T09:29:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.387+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac107ea0 con 0x7f6bac100ad0 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.389+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6ba80037a0 con 0x7f6bac100ad0 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.390+0000 7f6ba37fe640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 0x7f6b980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.390+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f6ba80cf710 con 0x7f6bac100ad0 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.390+0000 7f6bb199c640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 0x7f6b980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.390+0000 7f6bb199c640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 0x7f6b980a5700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06b050 tx=0x7f6b9c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.397+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ba809ccb0 con 0x7f6bac100ad0 2024-09-19T09:29:01.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:01.582+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6bac062630 con 0x7f6b980a3240 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:01.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:01.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:01 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]: dispatch 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:01.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:01 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:02.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:29:02.048+0000 7f8039e7b640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T09:29:02.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:29:02.062+0000 7f80324ad640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi028 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi028 to 4273M 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:02.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:02 smithi028 ceph-mon[30972]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' 2024-09-19T09:29:02.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi028 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi028 to 4273M 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:02 smithi078 ceph-mon[35619]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' 2024-09-19T09:29:02.613 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi028 2024-09-19T09:29:02.613 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi028 to 4273M 2024-09-19T09:29:02.613 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi028", "root=default"]}]': finished 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:02.614 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:02 smithi038 ceph-mon[35271]: from='osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948]' entity='osd.9' 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] boot 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T09:29:03.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:03 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:29:03.926 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] boot 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T09:29:03.927 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:03 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: osd.9 [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] boot 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T09:29:03.928 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:03 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:29:04.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:04 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/2273774956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:04 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:04 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]': finished 2024-09-19T09:29:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:04 smithi038 ceph-mon[35271]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:29:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:04.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:04 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/2273774956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:04 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:04 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]': finished 2024-09-19T09:29:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:04 smithi078 ceph-mon[35619]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:29:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:04 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/2273774956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:04 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]: dispatch 2024-09-19T09:29:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:04 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cc99ff-e67d-4b7a-b4da-e54e634584e6"}]': finished 2024-09-19T09:29:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:04 smithi028 ceph-mon[30972]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:29:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:05.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:05 smithi028 ceph-mon[30972]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:05 smithi028 ceph-mon[30972]: from='client.? 172.21.15.38:0/3515969983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:05.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:05 smithi078 ceph-mon[35619]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:05.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:05 smithi078 ceph-mon[35619]: from='client.? 172.21.15.38:0/3515969983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:05.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:05 smithi038 ceph-mon[35271]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:05.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:05 smithi038 ceph-mon[35271]: from='client.? 172.21.15.38:0/3515969983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:07.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:06 smithi078 ceph-mon[35619]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:07.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:06 smithi038 ceph-mon[35271]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:07.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:06 smithi028 ceph-mon[30972]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:09.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:08 smithi078 ceph-mon[35619]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:09.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:08 smithi038 ceph-mon[35271]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:08 smithi028 ceph-mon[30972]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:10.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:10 smithi038 ceph-mon[35271]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:11.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:10 smithi078 ceph-mon[35619]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:11.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:10 smithi028 ceph-mon[30972]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:13.053 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:12 smithi038 ceph-mon[35271]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:13.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:12 smithi078 ceph-mon[35619]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:13.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:12 smithi028 ceph-mon[30972]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:13.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:29:13.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:14.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:29:14.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:14.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:29:14.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:14.937 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:14 smithi038 ceph-mon[35271]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:14.938 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:14 smithi038 ceph-mon[35271]: Deploying daemon osd.10 on smithi038 2024-09-19T09:29:15.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:14 smithi078 ceph-mon[35619]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:15.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:14 smithi078 ceph-mon[35619]: Deploying daemon osd.10 on smithi038 2024-09-19T09:29:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:14 smithi028 ceph-mon[30972]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:15.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:14 smithi028 ceph-mon[30972]: Deploying daemon osd.10 on smithi038 2024-09-19T09:29:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:16 smithi038 ceph-mon[35271]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:17.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:16 smithi078 ceph-mon[35619]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:17.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:16 smithi028 ceph-mon[30972]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:19.071 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:18 smithi038 ceph-mon[35271]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:19.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:18 smithi078 ceph-mon[35619]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:19.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:18 smithi028 ceph-mon[30972]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:20.054 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:19 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:20.054 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:19 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.054 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:19 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:19 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:20.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:19 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:19 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:19 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:20.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:19 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:19 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:20.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:20 smithi038 ceph-mon[35271]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:21.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:20 smithi078 ceph-mon[35619]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:21.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:20 smithi028 ceph-mon[30972]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:22.669 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 10 on host 'smithi038' 2024-09-19T09:29:22.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.668+0000 7f6ba37fe640 1 -- 172.21.15.38:0/1281349241 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6bac062630 con 0x7f6b980a3240 2024-09-19T09:29:22.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 msgr2=0x7f6b980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 0x7f6b980a5700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06b050 tx=0x7f6b9c00b390 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 msgr2=0x7f6bac06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 0x7f6bac06aad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9c0 tx=0x7f6ba800ee80 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 shutdown_connections 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6b980a3240 0x7f6b980a5700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bac102690 0x7f6bac19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bac101cd0 0x7f6bac19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 --2- 172.21.15.38:0/1281349241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac100ad0 0x7f6bac06aad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 >> 172.21.15.38:0/1281349241 conn(0x7f6bac0fc280 msgr2=0x7f6bac105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 shutdown_connections 2024-09-19T09:29:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T09:29:22.670+0000 7f6bb4401640 1 -- 172.21.15.38:0/1281349241 wait complete. 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:23.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.272 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.273 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.302 DEBUG:teuthology.orchestra.run.smithi038:osd.10> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T09:29:23.304 INFO:tasks.cephadm:Deploying osd.11 on smithi078 with /dev/vg_nvme/lv_1... 2024-09-19T09:29:23.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.332 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:23.556 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:29:25.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:24 smithi028 ceph-mon[30972]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:25.354 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:24 smithi078 ceph-mon[35619]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:25.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:24 smithi038 ceph-mon[35271]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:26.000 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:29:25 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:29:25.677+0000 7f240be06640 -1 osd.10 0 log_to_monitors true 2024-09-19T09:29:26.095 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-19T09:29:26.122 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-09-19T09:29:26.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:25 smithi028 ceph-mon[30972]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:25 smithi028 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:25 smithi078 ceph-mon[35619]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:25 smithi078 ceph-mon[35619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:25 smithi038 ceph-mon[35271]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:25 smithi038 ceph-mon[35271]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:29:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.926+0000 7f59ad015640 1 -- 172.21.15.78:0/666505279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8102690 msgr2=0x7f59a8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.926+0000 7f59ad015640 1 --2- 172.21.15.78:0/666505279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8102690 0x7f59a8108f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 -- 172.21.15.78:0/666505279 shutdown_connections 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 --2- 172.21.15.78:0/666505279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8102690 0x7f59a8108f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 --2- 172.21.15.78:0/666505279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8101cd0 0x7f59a8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 --2- 172.21.15.78:0/666505279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 0x7f59a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 -- 172.21.15.78:0/666505279 >> 172.21.15.78:0/666505279 conn(0x7f59a80fc280 msgr2=0x7f59a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:26.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 -- 172.21.15.78:0/666505279 shutdown_connections 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 -- 172.21.15.78:0/666505279 wait complete. 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.927+0000 7f59ad015640 1 Processor -- start 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 -- start start 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 0x7f59a8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8102690 0x7f59a819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810d960 con 0x7f59a8102690 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810d7e0 con 0x7f59a8100ad0 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810dae0 con 0x7f59a8101cd0 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45138/0 (socket says 172.21.15.78:45138) 2024-09-19T09:29:26.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 -- 172.21.15.78:0/3100760207 learned_addr learned my addr 172.21.15.78:0/3100760207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a6d9d640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 0x7f59a8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 -- 172.21.15.78:0/3100760207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 msgr2=0x7f59a8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 0x7f59a8198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 -- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8102690 msgr2=0x7f59a819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a759e640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8102690 0x7f59a819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8102690 0x7f59a819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 -- 172.21.15.78:0/3100760207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a819fc50 con 0x7f59a8101cd0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59a659c640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f599c00b4a0 tx=0x7f599c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c014070 con 0x7f59a8101cd0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a819ff40 con 0x7f59a8101cd0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f599c010af0 con 0x7f59a8101cd0 2024-09-19T09:29:26.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.928+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c017d10 con 0x7f59a8101cd0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.929+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a81a0470 con 0x7f59a8101cd0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.929+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a8107f20 con 0x7f59a8101cd0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.930+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f599c017070 con 0x7f59a8101cd0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.931+0000 7f598ffff640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 0x7f59900a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.932+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6398+0+0 (secure 0 0 0) 0x7f599c0ce590 con 0x7f59a8101cd0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.932+0000 7f59a6d9d640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 0x7f59900a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.932+0000 7f59a6d9d640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 0x7f59900a5730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59a8101b30 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:26.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:26.936+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f599c09ba30 con 0x7f59a8101cd0 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi038 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:27.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:27.114+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f59a8062630 con 0x7f59900a3270 2024-09-19T09:29:27.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:27.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:29:27.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:29:27.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi038 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T09:29:27.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:27.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:27.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi038 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:27.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:28.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:28 smithi078 ceph-mon[35619]: from='osd.10 ' entity='osd.10' 2024-09-19T09:29:28.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:28.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:28 smithi038 ceph-mon[35271]: from='osd.10 ' entity='osd.10' 2024-09-19T09:29:28.177 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:29:27.839+0000 7f2407566640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T09:29:28.177 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:29:27 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:29:27.853+0000 7f23ffb98640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:29:28.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:29:28.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:29:28.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:28.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:28 smithi028 ceph-mon[30972]: from='osd.10 ' entity='osd.10' 2024-09-19T09:29:29.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] boot 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:29:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:29 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] boot 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:29:29.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:29 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: osd.10 [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] boot 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:29:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:29 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:29:30.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T09:29:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/3431311945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]': finished 2024-09-19T09:29:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:29:30.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:30 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:30.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T09:29:30.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/3431311945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]': finished 2024-09-19T09:29:30.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:29:30.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:30 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/3431311945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]: dispatch 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3515d175-e41e-41d0-80d3-bc30d0657a87"}]': finished 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:29:30.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:30 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:31.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:31 smithi038 ceph-mon[35271]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:31.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:31 smithi038 ceph-mon[35271]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:29:31.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:31.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:31 smithi038 ceph-mon[35271]: from='client.? 172.21.15.78:0/4024852411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:31.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:31 smithi028 ceph-mon[30972]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:31 smithi028 ceph-mon[30972]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:29:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:31 smithi028 ceph-mon[30972]: from='client.? 172.21.15.78:0/4024852411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:31.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:31 smithi078 ceph-mon[35619]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:31 smithi078 ceph-mon[35619]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:29:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:31 smithi078 ceph-mon[35619]: from='client.? 172.21.15.78:0/4024852411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:29:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:32 smithi038 ceph-mon[35271]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:29:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:32 smithi028 ceph-mon[30972]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:29:32.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:32 smithi078 ceph-mon[35619]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:29:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:33.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:33 smithi038 ceph-mon[35271]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:33.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:33 smithi028 ceph-mon[30972]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:33.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:33 smithi078 ceph-mon[35619]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:35.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:35 smithi078 ceph-mon[35619]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-19T09:29:35.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:35 smithi038 ceph-mon[35271]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-19T09:29:35.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:35 smithi028 ceph-mon[30972]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-19T09:29:37.186 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:37 smithi078 ceph-mon[35619]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:29:37.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:37 smithi038 ceph-mon[35271]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:29:37.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:37 smithi028 ceph-mon[30972]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:29:39.242 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:39 smithi078 ceph-mon[35619]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:29:39.242 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:29:39.243 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:39.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:39 smithi028 ceph-mon[30972]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:29:39.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:29:39.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:39.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:39 smithi038 ceph-mon[35271]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:29:39.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:29:39.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:40.432 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:40 smithi078 ceph-mon[35619]: Deploying daemon osd.11 on smithi078 2024-09-19T09:29:40.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:40 smithi028 ceph-mon[30972]: Deploying daemon osd.11 on smithi078 2024-09-19T09:29:40.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:40 smithi038 ceph-mon[35271]: Deploying daemon osd.11 on smithi078 2024-09-19T09:29:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:41 smithi078 ceph-mon[35619]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:29:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:41 smithi028 ceph-mon[30972]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:29:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:41 smithi038 ceph-mon[35271]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:29:43.301 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:43 smithi078 ceph-mon[35619]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:29:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:43 smithi028 ceph-mon[30972]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:29:43.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:43 smithi038 ceph-mon[35271]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:29:45.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:45 smithi078 ceph-mon[35619]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:29:45.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:45.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:45.329 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:45.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:45 smithi028 ceph-mon[30972]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:29:45.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:45.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:45.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:45 smithi038 ceph-mon[35271]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:29:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:45.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:47 smithi028 ceph-mon[30972]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:47.598 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:47 smithi078 ceph-mon[35619]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:47.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.636+0000 7f598ffff640 1 -- 172.21.15.78:0/3100760207 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f59a8062630 con 0x7f59900a3270 2024-09-19T09:29:47.637 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 11 on host 'smithi078' 2024-09-19T09:29:47.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 msgr2=0x7f59900a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 0x7f59900a5730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59a8101b30 tx=0x7f5998008040 comp rx=0 tx=0).stop 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 msgr2=0x7f59a8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f599c00b4a0 tx=0x7f599c00b960 comp rx=0 tx=0).stop 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 shutdown_connections 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59900a3270 0x7f59900a5730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a8102690 0x7f59a819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a8101cd0 0x7f59a8198560 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 --2- 172.21.15.78:0/3100760207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59a8100ad0 0x7f59a8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 >> 172.21.15.78:0/3100760207 conn(0x7f59a80fc280 msgr2=0x7f59a81050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 shutdown_connections 2024-09-19T09:29:47.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:29:47.638+0000 7f59ad015640 1 -- 172.21.15.78:0/3100760207 wait complete. 2024-09-19T09:29:47.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:47 smithi038 ceph-mon[35271]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:48.662 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:48 smithi078 ceph-mon[35619]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:48.662 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:48.662 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:48.694 DEBUG:teuthology.orchestra.run.smithi078:osd.11> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T09:29:48.696 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T09:29:48.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:29:48.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:48 smithi038 ceph-mon[35271]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:48.967 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:49.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:48 smithi028 ceph-mon[30972]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:49.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:49.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:49.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 -- 172.21.15.28:0/1771655661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90102e00 msgr2=0x7f4f90103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 --2- 172.21.15.28:0/1771655661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90102e00 0x7f4f90103280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009900 tx=0x7f4f8402a410 comp rx=0 tx=0).stop 2024-09-19T09:29:49.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 -- 172.21.15.28:0/1771655661 shutdown_connections 2024-09-19T09:29:49.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 --2- 172.21.15.28:0/1771655661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f901037c0 0x7f4f9010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 --2- 172.21.15.28:0/1771655661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90102e00 0x7f4f90103280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 --2- 172.21.15.28:0/1771655661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f90101c00 0x7f4f90102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 -- 172.21.15.28:0/1771655661 >> 172.21.15.28:0/1771655661 conn(0x7f4f900fd390 msgr2=0x7f4f900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 -- 172.21.15.28:0/1771655661 shutdown_connections 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.425+0000 7f4f95545640 1 -- 172.21.15.28:0/1771655661 wait complete. 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 Processor -- start 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 -- start start 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f90102e00 0x7f4f900745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f901037c0 0x7f4f90074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9010d860 con 0x7f4f90101c00 2024-09-19T09:29:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9010d6e0 con 0x7f4f901037c0 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f95545640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9010d9e0 con 0x7f4f90102e00 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60590/0 (socket says 172.21.15.28:60590) 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 -- 172.21.15.28:0/3754041222 learned_addr learned my addr 172.21.15.28:0/3754041222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8e7fc640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f90102e00 0x7f4f900745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8f7fe640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f901037c0 0x7f4f90074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 -- 172.21.15.28:0/3754041222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f90102e00 msgr2=0x7f4f900745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f90102e00 0x7f4f900745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 -- 172.21.15.28:0/3754041222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f901037c0 msgr2=0x7f4f90074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f901037c0 0x7f4f90074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.426+0000 7f4f8effd640 1 -- 172.21.15.28:0/3754041222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f900753e0 con 0x7f4f90101c00 2024-09-19T09:29:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f8effd640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c970 tx=0x7f4f8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80007c10 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f80007db0 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f901a1fe0 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000f940 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.427+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901a23d0 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.428+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f90108fd0 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.429+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4f8000fae0 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.430+0000 7f4f77fff640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 0x7f4f780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.430+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f4f800cfa50 con 0x7f4f90101c00 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.430+0000 7f4f8e7fc640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 0x7f4f780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.431+0000 7f4f8e7fc640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 0x7f4f780a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f840315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.433+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f8009ce70 con 0x7f4f90101c00 2024-09-19T09:29:49.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.609+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4f9002da30 con 0x7f4f90101c00 2024-09-19T09:29:49.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.609+0000 7f4f77fff640 1 -- 172.21.15.28:0/3754041222 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f4f8009c770 con 0x7f4f90101c00 2024-09-19T09:29:49.611 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:29:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.611+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 msgr2=0x7f4f780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.611+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 0x7f4f780a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f840315d0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.611+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 msgr2=0x7f4f90075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.611+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c970 tx=0x7f4f8000ce30 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 shutdown_connections 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4f780a3240 0x7f4f780a5700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f901037c0 0x7f4f90074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f90102e00 0x7f4f900745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 --2- 172.21.15.28:0/3754041222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f90101c00 0x7f4f90075f90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 >> 172.21.15.28:0/3754041222 conn(0x7f4f900fd390 msgr2=0x7f4f901061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 shutdown_connections 2024-09-19T09:29:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:49.612+0000 7f4f95545640 1 -- 172.21.15.28:0/3754041222 wait complete. 2024-09-19T09:29:49.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:49.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:49.903 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:49 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3754041222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:50.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:50.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:50.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:49 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3754041222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:50.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:50.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:50.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:49 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3754041222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:50.385 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726738168,"num_in_osds":12,"osd_in_since":1726738169,"num_remapped_pgs":0} 2024-09-19T09:29:51.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:50 smithi078 ceph-mon[35619]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:50 smithi078 ceph-mon[35619]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:50 smithi078 ceph-mon[35619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:29:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:29:50.712+0000 7f5cfd7ac640 -1 osd.11 0 log_to_monitors true 2024-09-19T09:29:51.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:50 smithi038 ceph-mon[35271]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:51.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:50 smithi038 ceph-mon[35271]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:50 smithi038 ceph-mon[35271]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:50 smithi028 ceph-mon[30972]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:50 smithi028 ceph-mon[30972]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:50 smithi028 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:29:51.386 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:29:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:51.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:29:51.780+0000 7f5cf8f0c640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T09:29:51.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:29:51.793+0000 7f5cf5546640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:29:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-mon[35619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:29:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-mon[35619]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:29:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-mon[35619]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:51 smithi078 ceph-mon[35619]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:52.086 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:51 smithi028 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:29:52.086 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:51 smithi028 ceph-mon[30972]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:29:52.086 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:51 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:52.087 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:51 smithi028 ceph-mon[30972]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:52.087 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:51 smithi028 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:52.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 -- 172.21.15.28:0/2893094451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 msgr2=0x7fd284103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/2893094451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd284103280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd278009900 tx=0x7fd27802a410 comp rx=0 tx=0).stop 2024-09-19T09:29:52.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 -- 172.21.15.28:0/2893094451 shutdown_connections 2024-09-19T09:29:52.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/2893094451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 0x7fd28410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/2893094451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd284103280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/2893094451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd284102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 -- 172.21.15.28:0/2893094451 >> 172.21.15.28:0/2893094451 conn(0x7fd2840fd390 msgr2=0x7fd2840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 -- 172.21.15.28:0/2893094451 shutdown_connections 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 -- 172.21.15.28:0/2893094451 wait complete. 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.084+0000 7fd28a6f0640 1 Processor -- start 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 -- start start 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd28419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 0x7fd2841a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410d860 con 0x7fd284102e00 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410d6e0 con 0x7fd2841037c0 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd28a6f0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410d9e0 con 0x7fd284101c00 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd2837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd28419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd2837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60596/0 (socket says 172.21.15.28:60596) 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd2837fe640 1 -- 172.21.15.28:0/451064469 learned_addr learned my addr 172.21.15.28:0/451064469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd288c8d640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 0x7fd2841a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 -- 172.21.15.28:0/451064469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 msgr2=0x7fd2841a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 0x7fd2841a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 -- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 msgr2=0x7fd28419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd283fff640 1 -- 172.21.15.28:0/451064469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2841a6100 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.085+0000 7fd2837fe640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd283fff640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd28419e4d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd27400e9c0 tx=0x7fd27400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400ccc0 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2740035f0 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841a63f0 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400ce60 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.086+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841a6900 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.087+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd284107090 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.088+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd2740037a0 con 0x7fd284101c00 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.089+0000 7fd2817fa640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 0x7fd26c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.089+0000 7fd2837fe640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 0x7fd26c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.089+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7fd2740cfb60 con 0x7fd284101c00 2024-09-19T09:29:52.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.089+0000 7fd2837fe640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 0x7fd26c0a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd278031440 tx=0x7fd2780315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.092+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd27409cda0 con 0x7fd284101c00 2024-09-19T09:29:52.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:51 smithi038 ceph-mon[35271]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:29:52.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:51 smithi038 ceph-mon[35271]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:29:52.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:51 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:52.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:51 smithi038 ceph-mon[35271]: from='osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:52.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:51 smithi038 ceph-mon[35271]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-19T09:29:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.271+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd284062df0 con 0x7fd284101c00 2024-09-19T09:29:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.271+0000 7fd2817fa640 1 -- 172.21.15.28:0/451064469 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fd27409c6a0 con 0x7fd284101c00 2024-09-19T09:29:52.273 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:29:52.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 msgr2=0x7fd26c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 0x7fd26c0a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd278031440 tx=0x7fd2780315b0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 msgr2=0x7fd28419e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd28419e4d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd27400e9c0 tx=0x7fd27400ee80 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 shutdown_connections 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd26c0a3270 0x7fd26c0a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2841037c0 0x7fd2841a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd284102e00 0x7fd28419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 --2- 172.21.15.28:0/451064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd284101c00 0x7fd28419e4d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 >> 172.21.15.28:0/451064469 conn(0x7fd2840fd390 msgr2=0x7fd284104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 shutdown_connections 2024-09-19T09:29:52.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:52.274+0000 7fd28a6f0640 1 -- 172.21.15.28:0/451064469 wait complete. 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: Detected new or changed devices on smithi078 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:52 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/451064469' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: Detected new or changed devices on smithi078 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.109 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.110 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:52 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/451064469' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:53.135 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726738168,"num_in_osds":12,"osd_in_since":1726738169,"num_remapped_pgs":0} 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: Detected new or changed devices on smithi078 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:29:53.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:52 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/451064469' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: purged_snaps scrub starts 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: purged_snaps scrub ok 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] boot 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:29:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:53 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.137 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:29:54.165 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: purged_snaps scrub starts 2024-09-19T09:29:54.166 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: purged_snaps scrub ok 2024-09-19T09:29:54.166 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.166 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] boot 2024-09-19T09:29:54.166 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:29:54.166 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:53 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: purged_snaps scrub starts 2024-09-19T09:29:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: purged_snaps scrub ok 2024-09-19T09:29:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: osd.11 [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] boot 2024-09-19T09:29:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:29:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:53 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:29:54.403 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 -- 172.21.15.28:0/2742058840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c1037c0 msgr2=0x7ff16c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 --2- 172.21.15.28:0/2742058840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c1037c0 0x7ff16c10a050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff164009ca0 tx=0x7ff16402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 -- 172.21.15.28:0/2742058840 shutdown_connections 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 --2- 172.21.15.28:0/2742058840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c102e00 0x7ff16c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 --2- 172.21.15.28:0/2742058840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c1037c0 0x7ff16c10a050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 --2- 172.21.15.28:0/2742058840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c101c00 0x7ff16c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 -- 172.21.15.28:0/2742058840 >> 172.21.15.28:0/2742058840 conn(0x7ff16c0fd390 msgr2=0x7ff16c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 -- 172.21.15.28:0/2742058840 shutdown_connections 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.014+0000 7ff17193a640 1 -- 172.21.15.28:0/2742058840 wait complete. 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 Processor -- start 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 -- start start 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c101c00 0x7ff16c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c1037c0 0x7ff16c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c10d900 con 0x7ff16c102e00 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c10d780 con 0x7ff16c101c00 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff17193a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c10da80 con 0x7ff16c1037c0 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34738/0 (socket says 172.21.15.28:34738) 2024-09-19T09:29:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 -- 172.21.15.28:0/3327903900 learned_addr learned my addr 172.21.15.28:0/3327903900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff16b7fe640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c1037c0 0x7ff16c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff16affd640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c101c00 0x7ff16c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 -- 172.21.15.28:0/3327903900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c1037c0 msgr2=0x7ff16c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c1037c0 0x7ff16c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 -- 172.21.15.28:0/3327903900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c101c00 msgr2=0x7ff16c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c101c00 0x7ff16c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.015+0000 7ff163fff640 1 -- 172.21.15.28:0/3327903900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c19fb20 con 0x7ff16c102e00 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff163fff640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c890 tx=0x7ff15400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154004220 con 0x7ff16c102e00 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1540043c0 con 0x7ff16c102e00 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c19fe10 con 0x7ff16c102e00 2024-09-19T09:29:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1540049e0 con 0x7ff16c102e00 2024-09-19T09:29:55.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.016+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1a0390 con 0x7ff16c102e00 2024-09-19T09:29:55.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.017+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16c108fb0 con 0x7ff16c102e00 2024-09-19T09:29:55.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.018+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff1540079b0 con 0x7ff16c102e00 2024-09-19T09:29:55.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.019+0000 7ff170938640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 0x7ff1580a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:55.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.019+0000 7ff16affd640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 0x7ff1580a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:55.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.019+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff1540cf1b0 con 0x7ff16c102e00 2024-09-19T09:29:55.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.019+0000 7ff16affd640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 0x7ff1580a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff16c102c60 tx=0x7ff15c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:55.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.023+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff154014030 con 0x7ff16c102e00 2024-09-19T09:29:55.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:54 smithi078 ceph-mon[35619]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:55.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:54 smithi078 ceph-mon[35619]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:29:55.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:54 smithi038 ceph-mon[35271]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:55.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:54 smithi038 ceph-mon[35271]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.199+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff16c062e90 con 0x7ff16c102e00 2024-09-19T09:29:55.201 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:54 smithi028 ceph-mon[30972]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:29:55.201 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:54 smithi028 ceph-mon[30972]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.200+0000 7ff170938640 1 -- 172.21.15.28:0/3327903900 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7ff15409c2d0 con 0x7ff16c102e00 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 msgr2=0x7ff1580a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 0x7ff1580a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff16c102c60 tx=0x7ff15c008040 comp rx=0 tx=0).stop 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 msgr2=0x7ff16c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c890 tx=0x7ff15400cd50 comp rx=0 tx=0).stop 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 shutdown_connections 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff1580a32f0 0x7ff1580a57b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff16c1037c0 0x7ff16c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff16c102e00 0x7ff16c198430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 --2- 172.21.15.28:0/3327903900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff16c101c00 0x7ff16c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 >> 172.21.15.28:0/3327903900 conn(0x7ff16c0fd390 msgr2=0x7ff16c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 shutdown_connections 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:55.202+0000 7ff17193a640 1 -- 172.21.15.28:0/3327903900 wait complete. 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726738192,"num_in_osds":12,"osd_in_since":1726738169,"num_remapped_pgs":0} 2024-09-19T09:29:55.946 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:29:56.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:55 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3327903900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:56.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:55 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3327903900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:56.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:55 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3327903900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 -- 172.21.15.28:0/391231146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 msgr2=0x7f83001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 --2- 172.21.15.28:0/391231146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83001021c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009900 tx=0x7f82f002a410 comp rx=0 tx=0).stop 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 -- 172.21.15.28:0/391231146 shutdown_connections 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 --2- 172.21.15.28:0/391231146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300102790 0x7f83001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 --2- 172.21.15.28:0/391231146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300106f10 0x7f830010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 --2- 172.21.15.28:0/391231146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83001021c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 -- 172.21.15.28:0/391231146 >> 172.21.15.28:0/391231146 conn(0x7f83000fd8c0 msgr2=0x7f83000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:56.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 -- 172.21.15.28:0/391231146 shutdown_connections 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.764+0000 7f830572a640 1 -- 172.21.15.28:0/391231146 wait complete. 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 Processor -- start 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 -- start start 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300102790 0x7f8300074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300106f10 0x7f8300074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8300111f90 con 0x7f8300101de0 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8300111e10 con 0x7f8300106f10 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f830572a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8300112110 con 0x7f8300102790 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f82feffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f82feffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34746/0 (socket says 172.21.15.28:34746) 2024-09-19T09:29:56.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f82feffd640 1 -- 172.21.15.28:0/747400498 learned_addr learned my addr 172.21.15.28:0/747400498 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:29:56.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f82ff7fe640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300106f10 0x7f8300074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:56.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.765+0000 7f82fe7fc640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300102790 0x7f8300074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 -- 172.21.15.28:0/747400498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300102790 msgr2=0x7f8300074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300102790 0x7f8300074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 -- 172.21.15.28:0/747400498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300106f10 msgr2=0x7f8300074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300106f10 0x7f8300074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 -- 172.21.15.28:0/747400498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83001a6940 con 0x7f8300101de0 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82feffd640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006010 tx=0x7f82f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f003f030 con 0x7f8300101de0 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f0003ce0 con 0x7f8300101de0 2024-09-19T09:29:56.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83001a6bd0 con 0x7f8300101de0 2024-09-19T09:29:56.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0035070 con 0x7f8300101de0 2024-09-19T09:29:56.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.766+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83001a70b0 con 0x7f8300101de0 2024-09-19T09:29:56.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.767+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83001021c0 con 0x7f8300101de0 2024-09-19T09:29:56.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.768+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f82f0042030 con 0x7f8300101de0 2024-09-19T09:29:56.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.769+0000 7f82e7fff640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 0x7f82e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:56.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.770+0000 7f82fe7fc640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 0x7f82e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:56.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.770+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f82f00efcc0 con 0x7f8300101de0 2024-09-19T09:29:56.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.770+0000 7f82fe7fc640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 0x7f82e80a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8300075f80 tx=0x7f82f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:56.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.773+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82f00bc850 con 0x7f8300101de0 2024-09-19T09:29:56.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.949+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f830010a0f0 con 0x7f8300101de0 2024-09-19T09:29:56.952 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:56 smithi028 ceph-mon[30972]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:56.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.950+0000 7f82e7fff640 1 -- 172.21.15.28:0/747400498 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18384 (secure 0 0 0) 0x7f82f00c0b50 con 0x7f8300101de0 2024-09-19T09:29:56.953 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:29:56.953 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:29:53.789027+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 msgr2=0x7f82e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 0x7f82e80a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8300075f80 tx=0x7f82f400b390 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 msgr2=0x7f83000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006010 tx=0x7f82f00042f0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 shutdown_connections 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f82e80a3240 0x7f82e80a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8300106f10 0x7f8300074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8300102790 0x7f8300074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 --2- 172.21.15.28:0/747400498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8300101de0 0x7f83000744f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 >> 172.21.15.28:0/747400498 conn(0x7f83000fd8c0 msgr2=0x7f83000fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:56.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 shutdown_connections 2024-09-19T09:29:56.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:56.953+0000 7f830572a640 1 -- 172.21.15.28:0/747400498 wait complete. 2024-09-19T09:29:57.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:56 smithi078 ceph-mon[35619]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:57.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:56 smithi038 ceph-mon[35271]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:57.737 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:26:05.815869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:29:57.737 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T09:29:57.996 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:58.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:57 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/747400498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:58.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:57 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/747400498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:58.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:57 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/747400498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 -- 172.21.15.28:0/4090452465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38106f10 msgr2=0x7efd3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 --2- 172.21.15.28:0/4090452465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38106f10 0x7efd3810ea90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd30009900 tx=0x7efd3002a410 comp rx=0 tx=0).stop 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 -- 172.21.15.28:0/4090452465 shutdown_connections 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 --2- 172.21.15.28:0/4090452465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38106f10 0x7efd3810ea90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 --2- 172.21.15.28:0/4090452465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38102790 0x7efd381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.516+0000 7efd403a3640 1 --2- 172.21.15.28:0/4090452465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38101de0 0x7efd381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- 172.21.15.28:0/4090452465 >> 172.21.15.28:0/4090452465 conn(0x7efd380fd8c0 msgr2=0x7efd380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- 172.21.15.28:0/4090452465 shutdown_connections 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- 172.21.15.28:0/4090452465 wait complete. 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 Processor -- start 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- start start 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38101de0 0x7efd380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38106f10 0x7efd38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efd38111f90 con 0x7efd38102790 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd38111e10 con 0x7efd38106f10 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.517+0000 7efd403a3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd38112110 con 0x7efd38101de0 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34766/0 (socket says 172.21.15.28:34766) 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 -- 172.21.15.28:0/3543579348 learned_addr learned my addr 172.21.15.28:0/3543579348 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3e940640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38106f10 0x7efd38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3e13f640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38101de0 0x7efd380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 -- 172.21.15.28:0/3543579348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38101de0 msgr2=0x7efd380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38101de0 0x7efd380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 -- 172.21.15.28:0/3543579348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38106f10 msgr2=0x7efd38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38106f10 0x7efd38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 -- 172.21.15.28:0/3543579348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd381a69c0 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd3d93e640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b4a0 tx=0x7efd2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28014070 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd381a6cb0 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd28010af0 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28017d10 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.518+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a71a0 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.519+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd381021c0 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.521+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efd28017070 con 0x7efd38102790 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.521+0000 7efd2f7fe640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.521+0000 7efd3e13f640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.521+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7efd280ce840 con 0x7efd38102790 2024-09-19T09:29:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.522+0000 7efd3e13f640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 0x7efd240a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:58.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.525+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efd2809ba00 con 0x7efd38102790 2024-09-19T09:29:58.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.701+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd3810a150 con 0x7efd38102790 2024-09-19T09:29:58.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.701+0000 7efd2f7fe640 1 -- 172.21.15.28:0/3543579348 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7efd2809b300 con 0x7efd38102790 2024-09-19T09:29:58.703 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-19T09:29:58.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.703+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 msgr2=0x7efd240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:58.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.703+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 0x7efd240a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34009870 comp rx=0 tx=0).stop 2024-09-19T09:29:58.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.703+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 msgr2=0x7efd38074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:58.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.703+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b4a0 tx=0x7efd2800b960 comp rx=0 tx=0).stop 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 shutdown_connections 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efd240a3240 0x7efd240a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd38106f10 0x7efd38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd38102790 0x7efd38074a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 --2- 172.21.15.28:0/3543579348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd38101de0 0x7efd380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 >> 172.21.15.28:0/3543579348 conn(0x7efd380fd8c0 msgr2=0x7efd380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 shutdown_connections 2024-09-19T09:29:58.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:29:58.704+0000 7efd403a3640 1 -- 172.21.15.28:0/3543579348 wait complete. 2024-09-19T09:29:59.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:58 smithi028 ceph-mon[30972]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:29:58 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3543579348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:58 smithi078 ceph-mon[35619]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:29:58 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3543579348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:59.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:58 smithi038 ceph-mon[35271]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:29:59.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:29:58 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3543579348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.a on smithi028 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.d on smithi028 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.g on smithi028 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.j on smithi028 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.b on smithi038 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.e on smithi038 2024-09-19T09:29:59.418 INFO:tasks.cephadm:Adding mds.h on smithi038 2024-09-19T09:29:59.419 INFO:tasks.cephadm:Adding mds.k on smithi038 2024-09-19T09:29:59.419 INFO:tasks.cephadm:Adding mds.c on smithi078 2024-09-19T09:29:59.419 INFO:tasks.cephadm:Adding mds.f on smithi078 2024-09-19T09:29:59.419 INFO:tasks.cephadm:Adding mds.i on smithi078 2024-09-19T09:29:59.419 INFO:tasks.cephadm:Adding mds.l on smithi078 2024-09-19T09:29:59.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l' 2024-09-19T09:29:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 -- 172.21.15.78:0/873288880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20102660 msgr2=0x7fdd20104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 --2- 172.21.15.78:0/873288880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20102660 0x7fdd20104a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009900 tx=0x7fdd1002a410 comp rx=0 tx=0).stop 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 -- 172.21.15.78:0/873288880 shutdown_connections 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 --2- 172.21.15.78:0/873288880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd20104f90 0x7fdd20107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 --2- 172.21.15.78:0/873288880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd201078c0 0x7fdd20109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 --2- 172.21.15.78:0/873288880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20102660 0x7fdd20104a50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 -- 172.21.15.78:0/873288880 >> 172.21.15.78:0/873288880 conn(0x7fdd200fc280 msgr2=0x7fdd200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 -- 172.21.15.78:0/873288880 shutdown_connections 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.158+0000 7fdd269dc640 1 -- 172.21.15.78:0/873288880 wait complete. 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 Processor -- start 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 -- start start 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd20102660 0x7fdd20197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd201078c0 0x7fdd2019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010d890 con 0x7fdd20102660 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010d710 con 0x7fdd201078c0 2024-09-19T09:30:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd269dc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010da10 con 0x7fdd20104f90 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34390/0 (socket says 172.21.15.78:34390) 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd1f7fe640 1 -- 172.21.15.78:0/1070110841 learned_addr learned my addr 172.21.15.78:0/1070110841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.159+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd20102660 0x7fdd20197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd24f79640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd201078c0 0x7fdd2019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 -- 172.21.15.78:0/1070110841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd201078c0 msgr2=0x7fdd2019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd201078c0 0x7fdd2019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 -- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd20102660 msgr2=0x7fdd20197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd20102660 0x7fdd20197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 -- 172.21.15.78:0/1070110841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2019fb50 con 0x7fdd20104f90 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1f7fe640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c970 tx=0x7fdd1400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14007bb0 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd14007d50 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140187a0 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2019fe40 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.160+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a02a0 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.161+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd200fdd70 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.162+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdd1400f450 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.163+0000 7fdd1d7fa640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 0x7fdd080a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.163+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fdd140cf4d0 con 0x7fdd20104f90 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.163+0000 7fdd1ffff640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 0x7fdd080a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.164+0000 7fdd1ffff640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 0x7fdd080a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd100314e0 tx=0x7fdd10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.167+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd14014030 con 0x7fdd20104f90 2024-09-19T09:30:00.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.346+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l", "target": ["mon-mgr", ""]}) -- 0x7fdd20062730 con 0x7fdd080a3270 2024-09-19T09:30:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.354+0000 7fdd1d7fa640 1 -- 172.21.15.78:0/1070110841 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdd20062730 con 0x7fdd080a3270 2024-09-19T09:30:00.355 INFO:teuthology.orchestra.run.smithi078.stdout:Scheduled mds.all update... 2024-09-19T09:30:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 msgr2=0x7fdd080a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 0x7fdd080a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd100314e0 tx=0x7fdd10000bc0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 msgr2=0x7fdd20198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c970 tx=0x7fdd1400ce30 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 shutdown_connections 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd080a3270 0x7fdd080a5730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd201078c0 0x7fdd2019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd20104f90 0x7fdd20198460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 --2- 172.21.15.78:0/1070110841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd20102660 0x7fdd20197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 >> 172.21.15.78:0/1070110841 conn(0x7fdd200fc280 msgr2=0x7fdd20102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 shutdown_connections 2024-09-19T09:30:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-19T09:30:00.356+0000 7fdd269dc640 1 -- 172.21.15.78:0/1070110841 wait complete. 2024-09-19T09:30:01.074 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:01.074 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: overall HEALTH_OK 2024-09-19T09:30:01.074 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.074 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:01.075 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:00 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.076 DEBUG:teuthology.orchestra.run.smithi028:mds.a> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T09:30:01.082 DEBUG:teuthology.orchestra.run.smithi028:mds.d> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T09:30:01.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:01.107 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: overall HEALTH_OK 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:01.108 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:00 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.109 DEBUG:teuthology.orchestra.run.smithi028:mds.g> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T09:30:01.151 DEBUG:teuthology.orchestra.run.smithi028:mds.j> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T09:30:01.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: overall HEALTH_OK 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:01.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:00 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:01.194 DEBUG:teuthology.orchestra.run.smithi038:mds.b> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T09:30:01.197 DEBUG:teuthology.orchestra.run.smithi038:mds.e> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T09:30:01.243 DEBUG:teuthology.orchestra.run.smithi038:mds.h> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T09:30:01.286 DEBUG:teuthology.orchestra.run.smithi038:mds.k> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T09:30:01.329 DEBUG:teuthology.orchestra.run.smithi078:mds.c> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T09:30:01.331 DEBUG:teuthology.orchestra.run.smithi078:mds.f> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T09:30:01.375 DEBUG:teuthology.orchestra.run.smithi078:mds.i> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T09:30:01.420 DEBUG:teuthology.orchestra.run.smithi078:mds.l> sudo journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T09:30:01.463 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T09:30:01.463 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:01.722 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:02.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:01 smithi028 ceph-mon[30972]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:02.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:01 smithi028 ceph-mon[30972]: Saving service mds.all spec with placement smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l;count:12 2024-09-19T09:30:02.106 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:01 smithi028 ceph-mon[30972]: Deploying daemon mds.l on smithi078 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.103+0000 7f3c5ec83640 1 -- 172.21.15.28:0/463931116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58102790 msgr2=0x7f3c581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.103+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/463931116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58102790 0x7f3c581067e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009900 tx=0x7f3c4402a410 comp rx=0 tx=0).stop 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 -- 172.21.15.28:0/463931116 shutdown_connections 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/463931116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58106f10 0x7f3c5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/463931116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58102790 0x7f3c581067e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/463931116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58101de0 0x7f3c581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 -- 172.21.15.28:0/463931116 >> 172.21.15.28:0/463931116 conn(0x7f3c580fd8c0 msgr2=0x7f3c580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 -- 172.21.15.28:0/463931116 shutdown_connections 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 -- 172.21.15.28:0/463931116 wait complete. 2024-09-19T09:30:02.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.104+0000 7f3c5ec83640 1 Processor -- start 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 -- start start 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58102790 0x7f3c58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 0x7f3c58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58111fb0 con 0x7f3c58106f10 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58111e30 con 0x7f3c58102790 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ec83640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58112130 con 0x7f3c58101de0 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:50868/0 (socket says 172.21.15.28:50868) 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 -- 172.21.15.28:0/1488619020 learned_addr learned my addr 172.21.15.28:0/1488619020 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5d220640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 0x7f3c58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:02.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c57fff640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58102790 0x7f3c58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 -- 172.21.15.28:0/1488619020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58102790 msgr2=0x7f3c58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58102790 0x7f3c58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 -- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 msgr2=0x7f3c58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 0x7f3c58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5ca1f640 1 -- 172.21.15.28:0/1488619020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c581a6970 con 0x7f3c58101de0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.105+0000 7f3c5d220640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 0x7f3c58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.106+0000 7f3c5ca1f640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00b4a0 tx=0x7f3c4c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.106+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c014070 con 0x7f3c58101de0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.106+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c581a6c60 con 0x7f3c58101de0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.106+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4c010af0 con 0x7f3c58101de0 2024-09-19T09:30:02.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.106+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c017d10 con 0x7f3c58101de0 2024-09-19T09:30:02.111 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:01 smithi078 ceph-mon[35619]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:02.111 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:01 smithi078 ceph-mon[35619]: Saving service mds.all spec with placement smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l;count:12 2024-09-19T09:30:02.111 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:01 smithi078 ceph-mon[35619]: Deploying daemon mds.l on smithi078 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.107+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c581a7170 con 0x7f3c58101de0 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.108+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c581021c0 con 0x7f3c58101de0 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.109+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3c4c017070 con 0x7f3c58101de0 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.109+0000 7f3c55ffb640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 0x7f3c480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.109+0000 7f3c57fff640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 0x7f3c480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:02.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.109+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3c4c0ce150 con 0x7f3c58101de0 2024-09-19T09:30:02.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.110+0000 7f3c57fff640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 0x7f3c480a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000c00 tx=0x7f3c440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:02.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.113+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c4c09acc0 con 0x7f3c58101de0 2024-09-19T09:30:02.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:01 smithi038 ceph-mon[35271]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:02.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:01 smithi038 ceph-mon[35271]: Saving service mds.all spec with placement smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l;count:12 2024-09-19T09:30:02.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:01 smithi038 ceph-mon[35271]: Deploying daemon mds.l on smithi078 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.325+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c5810a120 con 0x7f3c58101de0 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.325+0000 7f3c55ffb640 1 -- 172.21.15.28:0/1488619020 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f3c4c09efc0 con 0x7f3c58101de0 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":1,"btime":"2024-09-19T09:22:57:723574+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 1 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 msgr2=0x7f3c480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 0x7f3c480a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000c00 tx=0x7f3c440058d0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 msgr2=0x7f3c580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00b4a0 tx=0x7f3c4c00b960 comp rx=0 tx=0).stop 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 shutdown_connections 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c480a3240 0x7f3c480a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c58102790 0x7f3c58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c58106f10 0x7f3c58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 --2- 172.21.15.28:0/1488619020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101de0 0x7f3c580744f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 >> 172.21.15.28:0/1488619020 conn(0x7f3c580fd8c0 msgr2=0x7f3c580fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 shutdown_connections 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:02.328+0000 7f3c5ec83640 1 -- 172.21.15.28:0/1488619020 wait complete. 2024-09-19T09:30:02.831 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:02 smithi078 systemd[1]: Starting Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:03.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:02 smithi078 ceph-mon[35619]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:03.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:02 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1488619020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:03.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:02 smithi078 podman[65745]: 2024-09-19 09:30:02.791599253 +0000 UTC m=+0.015765159 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:03.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:02 smithi078 podman[65745]: 2024-09-19 09:30:02.928858005 +0000 UTC m=+0.153023894 container create 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:30:03.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:02 smithi038 ceph-mon[35271]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:03.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:02 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1488619020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:03.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:02 smithi028 ceph-mon[30972]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:03.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:02 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1488619020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 podman[65745]: 2024-09-19 09:30:03.173263356 +0000 UTC m=+0.397429250 container init 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 podman[65745]: 2024-09-19 09:30:03.176366793 +0000 UTC m=+0.400532693 container start 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 ceph-mds[65764]: -- 172.21.15.78:0/2362248549 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b22301b860 con 0x55b222ff9000 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[65760]: starting mds.l at 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 bash[65745]: 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 2024-09-19T09:30:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:30:03 smithi078 systemd[1]: Started Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:04.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6832/2992347158,v1:172.21.15.78:6833/2992347158] up:boot 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: fsmap 1 up:standby 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:04.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:04 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:04.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6832/2992347158,v1:172.21.15.78:6833/2992347158] up:boot 2024-09-19T09:30:04.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: fsmap 1 up:standby 2024-09-19T09:30:04.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:04.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:04.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:04.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:04 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:04.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6832/2992347158,v1:172.21.15.78:6833/2992347158] up:boot 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: fsmap 1 up:standby 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:04.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:04 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:05.037 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:05.414 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:05 smithi038 ceph-mon[35271]: Deploying daemon mds.k on smithi038 2024-09-19T09:30:05.414 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:05 smithi038 ceph-mon[35271]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:05.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:05 smithi028 ceph-mon[30972]: Deploying daemon mds.k on smithi038 2024-09-19T09:30:05.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:05 smithi028 ceph-mon[30972]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:05.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:05 smithi078 ceph-mon[35619]: Deploying daemon mds.k on smithi038 2024-09-19T09:30:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:05 smithi078 ceph-mon[35619]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:05.668 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:30:05 smithi038 podman[64963]: 2024-09-19 09:30:05.643849619 +0000 UTC m=+0.338965356 container start 53d58698adb10f08801b1389abdcd2bfdaf1b8ba10f1ef50cc3f50a4a94c9f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:05.668 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:30:05 smithi038 ceph-mds[64982]: -- 172.21.15.38:0/2910384794 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9b9e49860 con 0x55f9b9e27400 2024-09-19T09:30:05.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- 172.21.15.28:0/4241675032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c8074620 msgr2=0x7f39c8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:05.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/4241675032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c8074620 0x7f39c8074a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009900 tx=0x7f39c402a480 comp rx=0 tx=0).stop 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- 172.21.15.28:0/4241675032 shutdown_connections 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/4241675032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 0x7f39c810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/4241675032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c810e3c0 0x7f39c8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/4241675032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c8074620 0x7f39c8074a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- 172.21.15.28:0/4241675032 >> 172.21.15.28:0/4241675032 conn(0x7f39c80fd8c0 msgr2=0x7f39c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- 172.21.15.28:0/4241675032 shutdown_connections 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- 172.21.15.28:0/4241675032 wait complete. 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 Processor -- start 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- start start 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8074620 0x7f39c819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 0x7f39c819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81141c0 con 0x7f39c810e3c0 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114040 con 0x7f39c8074f40 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.736+0000 7f39cf9bb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8114340 con 0x7f39c8074620 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52018/0 (socket says 172.21.15.28:52018) 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 -- 172.21.15.28:0/1861670857 learned_addr learned my addr 172.21.15.28:0/1861670857 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cd757640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8074620 0x7f39c819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39ccf56640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 0x7f39c819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 -- 172.21.15.28:0/1861670857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8074620 msgr2=0x7f39c819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8074620 0x7f39c819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 -- 172.21.15.28:0/1861670857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 msgr2=0x7f39c819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 0x7f39c819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 -- 172.21.15.28:0/1861670857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c819f5f0 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cdf58640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b970 tx=0x7f39c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0019070 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c0014440 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c819f8e0 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00075d0 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.737+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a71c0 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.738+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c8074a00 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.740+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f39c001c070 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.740+0000 7f39be7fc640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 0x7f39b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.740+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f39c00ceec0 con 0x7f39c810e3c0 2024-09-19T09:30:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.740+0000 7f39cd757640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 0x7f39b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:05.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.741+0000 7f39cd757640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 0x7f39b40a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c00 tx=0x7f39c40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:05.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.744+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39c009ba30 con 0x7f39c810e3c0 2024-09-19T09:30:05.926 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:30:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: starting mds.k at 2024-09-19T09:30:05.926 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:30:05 smithi038 bash[64963]: 53d58698adb10f08801b1389abdcd2bfdaf1b8ba10f1ef50cc3f50a4a94c9f77 2024-09-19T09:30:05.926 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:30:05 smithi038 systemd[1]: Started Ceph mds.k for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:05.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.957+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39c802d7c0 con 0x7f39c810e3c0 2024-09-19T09:30:05.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.959+0000 7f39be7fc640 1 -- 172.21.15.28:0/1861670857 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f39c009fd30 con 0x7f39c810e3c0 2024-09-19T09:30:05.960 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:05.960 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":2,"btime":"2024-09-19T09:30:03:204540+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:30:05.960 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 2 2024-09-19T09:30:05.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 msgr2=0x7f39b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 0x7f39b40a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c00 tx=0x7f39c40058d0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 msgr2=0x7f39c819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b970 tx=0x7f39c000bd70 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 shutdown_connections 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39b40a3240 0x7f39b40a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39c8074f40 0x7f39c819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39c810e3c0 0x7f39c819ef10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 --2- 172.21.15.28:0/1861670857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8074620 0x7f39c819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 >> 172.21.15.28:0/1861670857 conn(0x7f39c80fd8c0 msgr2=0x7f39c80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 shutdown_connections 2024-09-19T09:30:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:05.961+0000 7f39cf9bb640 1 -- 172.21.15.28:0/1861670857 wait complete. 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: Deploying daemon mds.j on smithi028 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1861670857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6832/3013823435,v1:172.21.15.38:6833/3013823435] up:boot 2024-09-19T09:30:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: fsmap 2 up:standby 2024-09-19T09:30:07.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:07.112 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:07.112 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.112 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.112 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.112 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: Deploying daemon mds.j on smithi028 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1861670857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6832/3013823435,v1:172.21.15.38:6833/3013823435] up:boot 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: fsmap 2 up:standby 2024-09-19T09:30:07.113 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:07.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: Deploying daemon mds.j on smithi028 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1861670857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6832/3013823435,v1:172.21.15.38:6833/3013823435] up:boot 2024-09-19T09:30:07.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: fsmap 2 up:standby 2024-09-19T09:30:07.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:08.514 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 systemd[1]: Starting Ceph mds.j for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:08.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:08.871 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:08 smithi028 ceph-mon[30972]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:08.872 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 podman[69983]: 2024-09-19 09:30:08.60979684 +0000 UTC m=+0.108513396 container create 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:08.872 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 podman[69983]: 2024-09-19 09:30:08.518069573 +0000 UTC m=+0.016786094 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:08.872 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 podman[69983]: 2024-09-19 09:30:08.87102045 +0000 UTC m=+0.369736979 container init 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T09:30:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:09.049 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:08 smithi038 ceph-mon[35271]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:09.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:08 smithi078 ceph-mon[35619]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:09.134 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 podman[69983]: 2024-09-19 09:30:08.875022603 +0000 UTC m=+0.373739128 container start 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T09:30:09.134 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 ceph-mds[70027]: -- 172.21.15.28:0/3877574305 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adf62b3c20 con 0x55adf6290800 2024-09-19T09:30:09.134 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j[70016]: starting mds.j at 2024-09-19T09:30:09.135 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 bash[69983]: 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 2024-09-19T09:30:09.135 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:30:08 smithi028 systemd[1]: Started Ceph mds.j for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 -- 172.21.15.28:0/396010308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 msgr2=0x7fa524106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/396010308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa524106e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa520009930 tx=0x7fa52002a420 comp rx=0 tx=0).stop 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 -- 172.21.15.28:0/396010308 shutdown_connections 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/396010308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 0x7fa52410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/396010308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 0x7fa524100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/396010308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa524106e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 -- 172.21.15.28:0/396010308 >> 172.21.15.28:0/396010308 conn(0x7fa5240fc7a0 msgr2=0x7fa5240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 -- 172.21.15.28:0/396010308 shutdown_connections 2024-09-19T09:30:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.499+0000 7fa52bbf7640 1 -- 172.21.15.28:0/396010308 wait complete. 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 Processor -- start 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 -- start start 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 0x7fa524197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 0x7fa5241978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa524110e40 con 0x7fa524106a80 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa524110cc0 con 0x7fa524101420 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52bbf7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa524110fc0 con 0x7fa524100a80 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52a194640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52a194640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52044/0 (socket says 172.21.15.28:52044) 2024-09-19T09:30:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa52a194640 1 -- 172.21.15.28:0/3941196019 learned_addr learned my addr 172.21.15.28:0/3941196019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.500+0000 7fa529192640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 0x7fa5241978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa529993640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 0x7fa524197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 -- 172.21.15.28:0/3941196019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 msgr2=0x7fa524197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 0x7fa524197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 -- 172.21.15.28:0/3941196019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 msgr2=0x7fa5241978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 0x7fa5241978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 -- 172.21.15.28:0/3941196019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52419c330 con 0x7fa524106a80 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52a194640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00eaa0 tx=0x7fa51c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cdb0 con 0x7fa524106a80 2024-09-19T09:30:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51c005020 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52419c620 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0116b0 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.501+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5241a3f00 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.502+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa524100ee0 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.503+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa51c005560 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.504+0000 7fa51affd640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 0x7fa5100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.504+0000 7fa529993640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 0x7fa5100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.504+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa51c0cf700 con 0x7fa524106a80 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.505+0000 7fa529993640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 0x7fa5100a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa520005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.509+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa51c016030 con 0x7fa524106a80 2024-09-19T09:30:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.728+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa524109050 con 0x7fa524106a80 2024-09-19T09:30:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.729+0000 7fa51affd640 1 -- 172.21.15.28:0/3941196019 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7fa51c09c820 con 0x7fa524106a80 2024-09-19T09:30:09.730 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:09.730 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":3,"btime":"2024-09-19T09:30:06:208744+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:30:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 3 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 msgr2=0x7fa5100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 0x7fa5100a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa520005a20 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 msgr2=0x7fa52419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00eaa0 tx=0x7fa51c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 shutdown_connections 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa5100a3240 0x7fa5100a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524101420 0x7fa5241978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa524106a80 0x7fa52419bc50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 --2- 172.21.15.28:0/3941196019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524100a80 0x7fa524197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 >> 172.21.15.28:0/3941196019 conn(0x7fa5240fc7a0 msgr2=0x7fa5240fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 shutdown_connections 2024-09-19T09:30:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:09.731+0000 7fa52bbf7640 1 -- 172.21.15.28:0/3941196019 wait complete. 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: Deploying daemon mds.d on smithi028 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3941196019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6834/4241327209,v1:172.21.15.28:6835/4241327209] up:boot 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: fsmap 3 up:standby 2024-09-19T09:30:10.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:09 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:10.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:10.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: Deploying daemon mds.d on smithi028 2024-09-19T09:30:10.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3941196019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:10.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6834/4241327209,v1:172.21.15.28:6835/4241327209] up:boot 2024-09-19T09:30:10.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: fsmap 3 up:standby 2024-09-19T09:30:10.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:09 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: Deploying daemon mds.d on smithi028 2024-09-19T09:30:10.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3941196019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:10.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6834/4241327209,v1:172.21.15.28:6835/4241327209] up:boot 2024-09-19T09:30:10.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: fsmap 3 up:standby 2024-09-19T09:30:10.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:09 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:30:11.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:11 smithi028 ceph-mon[30972]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:11.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:11 smithi078 ceph-mon[35619]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:11.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:11 smithi038 ceph-mon[35271]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:12.014 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:11 smithi028 systemd[1]: Starting Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:12.432 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 podman[70396]: 2024-09-19 09:30:12.021358789 +0000 UTC m=+0.017309784 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 podman[70396]: 2024-09-19 09:30:12.120537217 +0000 UTC m=+0.116488192 container create 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 podman[70396]: 2024-09-19 09:30:12.331720026 +0000 UTC m=+0.327671019 container init 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 podman[70396]: 2024-09-19 09:30:12.334664778 +0000 UTC m=+0.330615753 container start 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 ceph-mds[70415]: -- 172.21.15.28:0/2903590165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635bc32f860 con 0x5635bc30d000 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[70411]: starting mds.d at 2024-09-19T09:30:12.433 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 bash[70396]: 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 2024-09-19T09:30:12.572 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:12.764 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:30:12 smithi028 systemd[1]: Started Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:12.830 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:13.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:13 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:13.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:13.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.227 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:13.228 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:13.228 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:13 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 -- 172.21.15.28:0/1541882259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d10102d30 msgr2=0x7f9d10103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 --2- 172.21.15.28:0/1541882259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d10102d30 0x7f9d10103110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009930 tx=0x7f9d0002a420 comp rx=0 tx=0).stop 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 -- 172.21.15.28:0/1541882259 shutdown_connections 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 --2- 172.21.15.28:0/1541882259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101036e0 0x7f9d10107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 --2- 172.21.15.28:0/1541882259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d101082a0 0x7f9d1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 --2- 172.21.15.28:0/1541882259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d10102d30 0x7f9d10103110 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 -- 172.21.15.28:0/1541882259 >> 172.21.15.28:0/1541882259 conn(0x7f9d100fc7d0 msgr2=0x7f9d100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 -- 172.21.15.28:0/1541882259 shutdown_connections 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 -- 172.21.15.28:0/1541882259 wait complete. 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.310+0000 7f9d16163640 1 Processor -- start 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 -- start start 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d10102d30 0x7f9d10198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101082a0 0x7f9d1019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101120a0 con 0x7f9d101036e0 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10111f20 con 0x7f9d101082a0 2024-09-19T09:30:13.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d16163640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10112220 con 0x7f9d10102d30 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60802/0 (socket says 172.21.15.28:60802) 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 -- 172.21.15.28:0/3893160506 learned_addr learned my addr 172.21.15.28:0/3893160506 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0f7fe640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d10102d30 0x7f9d10198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0ffff640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101082a0 0x7f9d1019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 -- 172.21.15.28:0/3893160506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d10102d30 msgr2=0x7f9d10198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d10102d30 0x7f9d10198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 -- 172.21.15.28:0/3893160506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101082a0 msgr2=0x7f9d1019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101082a0 0x7f9d1019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.311+0000 7f9d0effd640 1 -- 172.21.15.28:0/3893160506 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1019d510 con 0x7f9d101036e0 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d0effd640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9f0 tx=0x7f9d0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:13.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400ccc0 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d04010910 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1019d800 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04018920 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.312+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a50e0 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.313+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1006a850 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.314+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9d04010ac0 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.315+0000 7f9d0cff9640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 0x7f9cf80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.315+0000 7f9d0f7fe640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 0x7f9cf80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.315+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9d040cf830 con 0x7f9d101036e0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.315+0000 7f9d0f7fe640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 0x7f9cf80a5780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009900 tx=0x7f9d000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.319+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d0409c950 con 0x7f9d101036e0 2024-09-19T09:30:13.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:13.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:13 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:13.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.533+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d1010a2b0 con 0x7f9d101036e0 2024-09-19T09:30:13.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.534+0000 7f9d0cff9640 1 -- 172.21.15.28:0/3893160506 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f9d0409c250 con 0x7f9d101036e0 2024-09-19T09:30:13.535 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:13.535 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":5,"btime":"2024-09-19T09:30:13:013571+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:30:13.535 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 5 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 msgr2=0x7f9cf80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 0x7f9cf80a5780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009900 tx=0x7f9d000058d0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 msgr2=0x7f9d10198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9f0 tx=0x7f9d0400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 shutdown_connections 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9cf80a32c0 0x7f9cf80a5780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d101082a0 0x7f9d1019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d101036e0 0x7f9d10198aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 --2- 172.21.15.28:0/3893160506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d10102d30 0x7f9d10198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 >> 172.21.15.28:0/3893160506 conn(0x7f9d100fc7d0 msgr2=0x7f9d10105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 shutdown_connections 2024-09-19T09:30:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:13.536+0000 7f9d16163640 1 -- 172.21.15.28:0/3893160506 wait complete. 2024-09-19T09:30:14.098 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:14 smithi028 ceph-mon[30972]: Deploying daemon mds.e on smithi038 2024-09-19T09:30:14.098 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:14 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6836/558171377,v1:172.21.15.28:6837/558171377] up:boot 2024-09-19T09:30:14.098 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:14 smithi028 ceph-mon[30972]: fsmap 4 up:standby 2024-09-19T09:30:14.098 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:14 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:30:14.098 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3893160506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:14.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mon[35271]: Deploying daemon mds.e on smithi038 2024-09-19T09:30:14.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6836/558171377,v1:172.21.15.28:6837/558171377] up:boot 2024-09-19T09:30:14.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mon[35271]: fsmap 4 up:standby 2024-09-19T09:30:14.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:30:14.162 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3893160506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:14.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:14 smithi078 ceph-mon[35619]: Deploying daemon mds.e on smithi038 2024-09-19T09:30:14.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:14 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6836/558171377,v1:172.21.15.28:6837/558171377] up:boot 2024-09-19T09:30:14.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:14 smithi078 ceph-mon[35619]: fsmap 4 up:standby 2024-09-19T09:30:14.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:14 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:30:14.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3893160506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:14.426 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 systemd[1]: Starting Ceph mds.e for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:14.805 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 podman[65275]: 2024-09-19 09:30:14.542933551 +0000 UTC m=+0.108053525 container create 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T09:30:14.806 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 podman[65275]: 2024-09-19 09:30:14.450828984 +0000 UTC m=+0.015948933 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:14.806 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 podman[65275]: 2024-09-19 09:30:14.779013536 +0000 UTC m=+0.344133478 container init 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T09:30:14.806 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 podman[65275]: 2024-09-19 09:30:14.781908593 +0000 UTC m=+0.347028525 container start 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:15 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-mds[65294]: -- 172.21.15.38:0/3852883697 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a90fd57860 con 0x55a90fd34800 2024-09-19T09:30:15.176 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: starting mds.e at 2024-09-19T09:30:15.177 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 bash[65275]: 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 2024-09-19T09:30:15.177 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:30:14 smithi038 systemd[1]: Started Ceph mds.e for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:15 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:15 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:16.130 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:16.339 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:16 smithi078 ceph-mon[35619]: Deploying daemon mds.f on smithi078 2024-09-19T09:30:16.339 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:16 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6834/347872214,v1:172.21.15.38:6835/347872214] up:boot 2024-09-19T09:30:16.339 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:16 smithi078 ceph-mon[35619]: fsmap 5 up:standby 2024-09-19T09:30:16.340 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:16 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:16.401 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:16.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:16 smithi038 ceph-mon[35271]: Deploying daemon mds.f on smithi078 2024-09-19T09:30:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:16 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6834/347872214,v1:172.21.15.38:6835/347872214] up:boot 2024-09-19T09:30:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:16 smithi038 ceph-mon[35271]: fsmap 5 up:standby 2024-09-19T09:30:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:16 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:16 smithi028 ceph-mon[30972]: Deploying daemon mds.f on smithi078 2024-09-19T09:30:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:16 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6834/347872214,v1:172.21.15.38:6835/347872214] up:boot 2024-09-19T09:30:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:16 smithi028 ceph-mon[30972]: fsmap 5 up:standby 2024-09-19T09:30:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:16 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 -- 172.21.15.28:0/1927391565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 msgr2=0x7f2ec010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/1927391565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec010ea90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ca0 tx=0x7f2eb802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 -- 172.21.15.28:0/1927391565 shutdown_connections 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/1927391565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec010ea90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/1927391565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0102790 0x7f2ec01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/1927391565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0101de0 0x7f2ec01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 -- 172.21.15.28:0/1927391565 >> 172.21.15.28:0/1927391565 conn(0x7f2ec00fd8c0 msgr2=0x7f2ec00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 -- 172.21.15.28:0/1927391565 shutdown_connections 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.949+0000 7f2ec60ba640 1 -- 172.21.15.28:0/1927391565 wait complete. 2024-09-19T09:30:16.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 Processor -- start 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 -- start start 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0102790 0x7f2ec0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0111f70 con 0x7f2ec0106f10 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0111df0 con 0x7f2ec0102790 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.950+0000 7f2ec60ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01120f0 con 0x7f2ec0101de0 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:58936/0 (socket says 172.21.15.28:58936) 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 -- 172.21.15.28:0/2006141417 learned_addr learned my addr 172.21.15.28:0/2006141417 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebffff640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebeffd640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0102790 0x7f2ec0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 -- 172.21.15.28:0/2006141417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0102790 msgr2=0x7f2ec0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0102790 0x7f2ec0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 -- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 msgr2=0x7f2ec0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 -- 172.21.15.28:0/2006141417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec01a6920 con 0x7f2ec0101de0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebffff640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebf7fe640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000d950 tx=0x7f2eb000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0017070 con 0x7f2ec0101de0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb0009c30 con 0x7f2ec0101de0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01a6c10 con 0x7f2ec0101de0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000f6b0 con 0x7f2ec0101de0 2024-09-19T09:30:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.951+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a7150 con 0x7f2ec0101de0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.952+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec01021c0 con 0x7f2ec0101de0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.954+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2eb00109d0 con 0x7f2ec0101de0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.954+0000 7f2ebcff9640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 0x7f2ea80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.954+0000 7f2ebeffd640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 0x7f2ea80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.954+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f2eb00ceff0 con 0x7f2ec0101de0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.955+0000 7f2ebeffd640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 0x7f2ea80a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ec01087b0 tx=0x7f2eb400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:16.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:16.957+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2eb0009de0 con 0x7f2ec0101de0 2024-09-19T09:30:17.019 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:16 smithi078 systemd[1]: Starting Ceph mds.f for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.171+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec010a0f0 con 0x7f2ec0101de0 2024-09-19T09:30:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.171+0000 7f2ebcff9640 1 -- 172.21.15.28:0/2006141417 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7f2eb009bb40 con 0x7f2ec0101de0 2024-09-19T09:30:17.173 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:17.174 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":6,"btime":"2024-09-19T09:30:15:028025+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:30:17.174 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 6 2024-09-19T09:30:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 msgr2=0x7f2ea80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 0x7f2ea80a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ec01087b0 tx=0x7f2eb400b390 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 msgr2=0x7f2ec00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000d950 tx=0x7f2eb000de10 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 shutdown_connections 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2ea80a3240 0x7f2ea80a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec0102790 0x7f2ec0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec0106f10 0x7f2ec0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 --2- 172.21.15.28:0/2006141417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec0101de0 0x7f2ec00744f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 >> 172.21.15.28:0/2006141417 conn(0x7f2ec00fd8c0 msgr2=0x7f2ec00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 shutdown_connections 2024-09-19T09:30:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:17.174+0000 7f2ec60ba640 1 -- 172.21.15.28:0/2006141417 wait complete. 2024-09-19T09:30:17.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:17 smithi028 ceph-mon[30972]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:17.311 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:17 smithi078 ceph-mon[35619]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:17.312 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 podman[66048]: 2024-09-19 09:30:16.92039873 +0000 UTC m=+0.015876818 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:17.312 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 podman[66048]: 2024-09-19 09:30:17.058260703 +0000 UTC m=+0.153751222 container create 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T09:30:17.312 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 podman[66048]: 2024-09-19 09:30:17.31119333 +0000 UTC m=+0.406671420 container init 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:17.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:17 smithi038 ceph-mon[35271]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:17.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 podman[66048]: 2024-09-19 09:30:17.314144493 +0000 UTC m=+0.409622587 container start 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T09:30:17.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 ceph-mds[66067]: -- 172.21.15.78:0/581295140 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583470e1860 con 0x5583470be800 2024-09-19T09:30:17.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: starting mds.f at 2024-09-19T09:30:17.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 bash[66048]: 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 2024-09-19T09:30:17.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:30:17 smithi078 systemd[1]: Started Ceph mds.f for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:18.199 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2006141417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:18.199 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.199 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.199 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.200 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:18.200 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:18.200 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:18 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2006141417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:18.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:18 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2006141417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:18.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:18 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:19.290 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mon[35619]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:19.290 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mon[35619]: Deploying daemon mds.c on smithi078 2024-09-19T09:30:19.290 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:boot 2024-09-19T09:30:19.290 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mon[35619]: fsmap 6 up:standby 2024-09-19T09:30:19.290 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:19.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:19 smithi038 ceph-mon[35271]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:19.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:19 smithi038 ceph-mon[35271]: Deploying daemon mds.c on smithi078 2024-09-19T09:30:19.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:19 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:boot 2024-09-19T09:30:19.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:19 smithi038 ceph-mon[35271]: fsmap 6 up:standby 2024-09-19T09:30:19.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:19 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:19.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:19 smithi028 ceph-mon[30972]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:19.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:19 smithi028 ceph-mon[30972]: Deploying daemon mds.c on smithi078 2024-09-19T09:30:19.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:19 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:boot 2024-09-19T09:30:19.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:19 smithi028 ceph-mon[30972]: fsmap 6 up:standby 2024-09-19T09:30:19.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:19 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:19.607 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 podman[66350]: 2024-09-19 09:30:19.504194 +0000 UTC m=+0.115923088 container create cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T09:30:19.816 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:20.086 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 podman[66350]: 2024-09-19 09:30:19.740201686 +0000 UTC m=+0.351930740 container init cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 podman[66350]: 2024-09-19 09:30:19.742983493 +0000 UTC m=+0.354712541 container start cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-mds[66369]: -- 172.21.15.78:0/2804572120 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558699a87860 con 0x558699a65000 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: starting mds.c at 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 bash[66350]: cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 2024-09-19T09:30:20.107 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:30:19 smithi078 systemd[1]: Started Ceph mds.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 -- 172.21.15.28:0/393478150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102ef0 msgr2=0x7f03d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 --2- 172.21.15.28:0/393478150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102ef0 0x7f03d410da40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009ca0 tx=0x7f03cc02a950 comp rx=0 tx=0).stop 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 -- 172.21.15.28:0/393478150 shutdown_connections 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 --2- 172.21.15.28:0/393478150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102570 0x7f03d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 --2- 172.21.15.28:0/393478150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102ef0 0x7f03d410da40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 --2- 172.21.15.28:0/393478150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 0x7f03d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 -- 172.21.15.28:0/393478150 >> 172.21.15.28:0/393478150 conn(0x7f03d40fc7d0 msgr2=0x7f03d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 -- 172.21.15.28:0/393478150 shutdown_connections 2024-09-19T09:30:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.547+0000 7f03dab78640 1 -- 172.21.15.28:0/393478150 wait complete. 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 Processor -- start 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 -- start start 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 0x7f03d4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102ef0 0x7f03d4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4110f80 con 0x7f03d4102570 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4110e00 con 0x7f03d4102ef0 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03dab78640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4111100 con 0x7f03d4100e20 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60846/0 (socket says 172.21.15.28:60846) 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03d3fff640 1 -- 172.21.15.28:0/3761108003 learned_addr learned my addr 172.21.15.28:0/3761108003 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03d8914640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 0x7f03d4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.548+0000 7f03d9115640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102ef0 0x7f03d4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 -- 172.21.15.28:0/3761108003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 msgr2=0x7f03d4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 0x7f03d4197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 -- 172.21.15.28:0/3761108003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102ef0 msgr2=0x7f03d4197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102ef0 0x7f03d4197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 -- 172.21.15.28:0/3761108003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d41a6a10 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d3fff640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03c000d950 tx=0x7f03c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0017070 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d41a6d00 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c0009c30 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000f6b0 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.549+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d41a71e0 con 0x7f03d4102570 2024-09-19T09:30:20.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.550+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d4101200 con 0x7f03d4102570 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.552+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f03c00109d0 con 0x7f03d4102570 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.552+0000 7f03d1ffb640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 0x7f03c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.552+0000 7f03d8914640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 0x7f03c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.552+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f03c00ceff0 con 0x7f03d4102570 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.553+0000 7f03d8914640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 0x7f03c40a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f03c800bbe0 tx=0x7f03c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:20.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.560+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03c0009de0 con 0x7f03d4102570 2024-09-19T09:30:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.775+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03d406a5a0 con 0x7f03d4102570 2024-09-19T09:30:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.776+0000 7f03d1ffb640 1 -- 172.21.15.28:0/3761108003 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6379 (secure 0 0 0) 0x7f03c009bae0 con 0x7f03d4102570 2024-09-19T09:30:20.778 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:20.778 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"btime":"2024-09-19T09:30:20:054318+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T09:30:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-09-19T09:30:20.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 msgr2=0x7f03c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 0x7f03c40a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f03c800bbe0 tx=0x7f03c800b390 comp rx=0 tx=0).stop 2024-09-19T09:30:20.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 msgr2=0x7f03d41976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03c000d950 tx=0x7f03c000de10 comp rx=0 tx=0).stop 2024-09-19T09:30:20.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 shutdown_connections 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f03c40a3240 0x7f03c40a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03d4102ef0 0x7f03d4197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d4102570 0x7f03d41976a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 --2- 172.21.15.28:0/3761108003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d4100e20 0x7f03d4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.778+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 >> 172.21.15.28:0/3761108003 conn(0x7f03d40fc7d0 msgr2=0x7f03d41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.779+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 shutdown_connections 2024-09-19T09:30:20.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:20.779+0000 7f03dab78640 1 -- 172.21.15.28:0/3761108003 wait complete. 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: Deploying daemon mds.h on smithi038 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:boot 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: fsmap 7 up:standby 2024-09-19T09:30:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:21.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:20 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3761108003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:21.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: Deploying daemon mds.h on smithi038 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:boot 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: fsmap 7 up:standby 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:21.177 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:20 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3761108003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: Deploying daemon mds.h on smithi038 2024-09-19T09:30:21.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:boot 2024-09-19T09:30:21.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: fsmap 7 up:standby 2024-09-19T09:30:21.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:21.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:20 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3761108003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.926 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:21 smithi038 systemd[1]: Starting Ceph mds.h for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:22.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 podman[65579]: 2024-09-19 09:30:22.0311475 +0000 UTC m=+0.108434018 container create fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T09:30:22.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 podman[65579]: 2024-09-19 09:30:21.938376541 +0000 UTC m=+0.015663067 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:22.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 podman[65579]: 2024-09-19 09:30:22.258859231 +0000 UTC m=+0.336145750 container init fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:22.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 podman[65579]: 2024-09-19 09:30:22.261642965 +0000 UTC m=+0.338929491 container start fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:22.289 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mds[65598]: -- 172.21.15.38:0/1986145286 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651b1411860 con 0x5651b13ef400 2024-09-19T09:30:22.675 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: starting mds.h at 2024-09-19T09:30:22.676 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 bash[65579]: fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 2024-09-19T09:30:22.676 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:30:22 smithi038 systemd[1]: Started Ceph mds.h for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:23.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:23.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:22 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:23.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:23.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:22 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:23.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:23.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:22 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:23.465 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:24.046 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:23 smithi078 ceph-mon[35619]: Deploying daemon mds.i on smithi078 2024-09-19T09:30:24.046 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:23 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:boot 2024-09-19T09:30:24.046 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:23 smithi078 ceph-mon[35619]: fsmap 8 up:standby 2024-09-19T09:30:24.046 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:23 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:24.140 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:23 smithi028 ceph-mon[30972]: Deploying daemon mds.i on smithi078 2024-09-19T09:30:24.141 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:23 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:boot 2024-09-19T09:30:24.141 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:23 smithi028 ceph-mon[30972]: fsmap 8 up:standby 2024-09-19T09:30:24.141 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:23 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:24.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:23 smithi038 ceph-mon[35271]: Deploying daemon mds.i on smithi078 2024-09-19T09:30:24.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:23 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:boot 2024-09-19T09:30:24.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:23 smithi038 ceph-mon[35271]: fsmap 8 up:standby 2024-09-19T09:30:24.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:23 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:24.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 -- 172.21.15.28:0/2302459653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec106fc0 msgr2=0x7f7dec10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 --2- 172.21.15.28:0/2302459653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec106fc0 0x7f7dec10eaf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009cd0 tx=0x7f7de402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 -- 172.21.15.28:0/2302459653 shutdown_connections 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 --2- 172.21.15.28:0/2302459653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100d20 0x7f7dec106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 --2- 172.21.15.28:0/2302459653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec106fc0 0x7f7dec10eaf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 --2- 172.21.15.28:0/2302459653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100370 0x7f7dec100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.221+0000 7f7df427c640 1 -- 172.21.15.28:0/2302459653 >> 172.21.15.28:0/2302459653 conn(0x7f7dec077210 msgr2=0x7f7dec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.222+0000 7f7df427c640 1 -- 172.21.15.28:0/2302459653 shutdown_connections 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.222+0000 7f7df427c640 1 -- 172.21.15.28:0/2302459653 wait complete. 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 Processor -- start 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 -- start start 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 0x7f7dec1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec106fc0 0x7f7dec101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec112000 con 0x7f7dec100d20 2024-09-19T09:30:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec111e80 con 0x7f7dec100370 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df427c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec112180 con 0x7f7dec106fc0 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df1817640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df1817640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49646/0 (socket says 172.21.15.28:49646) 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df1817640 1 -- 172.21.15.28:0/837682232 learned_addr learned my addr 172.21.15.28:0/837682232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df2018640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 0x7f7dec1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df2819640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec106fc0 0x7f7dec101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.223+0000 7f7df1817640 1 -- 172.21.15.28:0/837682232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec106fc0 msgr2=0x7f7dec101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df1817640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec106fc0 0x7f7dec101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df1817640 1 -- 172.21.15.28:0/837682232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 msgr2=0x7f7dec1012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df1817640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 0x7f7dec1012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df1817640 1 -- 172.21.15.28:0/837682232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec1a6920 con 0x7f7dec100d20 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df2018640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 0x7f7dec1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:24.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df1817640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00b450 tx=0x7f7ddc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc015020 con 0x7f7dec100d20 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ddc011d70 con 0x7f7dec100d20 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec1a6c10 con 0x7f7dec100d20 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00f340 con 0x7f7dec100d20 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.224+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a71a0 con 0x7f7dec100d20 2024-09-19T09:30:24.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.225+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec100750 con 0x7f7dec100d20 2024-09-19T09:30:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.226+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7ddc018020 con 0x7f7dec100d20 2024-09-19T09:30:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.227+0000 7f7de37fe640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 0x7f7dd80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.227+0000 7f7df2018640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 0x7f7dd80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.227+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7ddc0cefb0 con 0x7f7dec100d20 2024-09-19T09:30:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.227+0000 7f7df2018640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 0x7f7dd80a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7de80058c0 tx=0x7f7de8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:24.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.231+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7ddc01f020 con 0x7f7dec100d20 2024-09-19T09:30:24.357 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 systemd[1]: Starting Ceph mds.i for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:24.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.442+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dec102820 con 0x7f7dec100d20 2024-09-19T09:30:24.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.443+0000 7f7de37fe640 1 -- 172.21.15.28:0/837682232 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7201 (secure 0 0 0) 0x7f7dec102820 con 0x7f7dec100d20 2024-09-19T09:30:24.445 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:24.445 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-19T09:30:22:882506+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T09:30:24.445 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 msgr2=0x7f7dd80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 0x7f7dd80a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7de80058c0 tx=0x7f7de8005850 comp rx=0 tx=0).stop 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 msgr2=0x7f7dec101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00b450 tx=0x7f7ddc00b910 comp rx=0 tx=0).stop 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 shutdown_connections 2024-09-19T09:30:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a32c0 0x7f7dd80a5780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec106fc0 0x7f7dec101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec100d20 0x7f7dec101830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 --2- 172.21.15.28:0/837682232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec100370 0x7f7dec1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 >> 172.21.15.28:0/837682232 conn(0x7f7dec077210 msgr2=0x7f7dec0fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 shutdown_connections 2024-09-19T09:30:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:24.445+0000 7f7df427c640 1 -- 172.21.15.28:0/837682232 wait complete. 2024-09-19T09:30:24.686 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 podman[66648]: 2024-09-19 09:30:24.337411928 +0000 UTC m=+0.015645600 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:24.686 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 podman[66648]: 2024-09-19 09:30:24.466383281 +0000 UTC m=+0.144616942 container create f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T09:30:25.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/837682232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 podman[66648]: 2024-09-19 09:30:24.685564993 +0000 UTC m=+0.363798664 container init f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T09:30:25.107 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 podman[66648]: 2024-09-19 09:30:24.688635295 +0000 UTC m=+0.366868965 container start f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T09:30:25.108 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-mds[66667]: -- 172.21.15.78:0/3269040885 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614fc335860 con 0x5614fc313400 2024-09-19T09:30:25.108 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: starting mds.i at 2024-09-19T09:30:25.108 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 bash[66648]: f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 2024-09-19T09:30:25.108 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:24 smithi078 systemd[1]: Started Ceph mds.i for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:25.144 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:25.144 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/837682232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:25.144 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.144 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.144 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.145 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:25.145 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:25.145 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:24 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/837682232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:25.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:24 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:26.175 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:25 smithi028 ceph-mon[30972]: Deploying daemon mds.g on smithi028 2024-09-19T09:30:26.175 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:25 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:boot 2024-09-19T09:30:26.175 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:25 smithi028 ceph-mon[30972]: fsmap 9 up:standby 2024-09-19T09:30:26.176 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:25 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:25 smithi038 ceph-mon[35271]: Deploying daemon mds.g on smithi028 2024-09-19T09:30:26.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:25 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:boot 2024-09-19T09:30:26.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:25 smithi038 ceph-mon[35271]: fsmap 9 up:standby 2024-09-19T09:30:26.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:25 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:25 smithi078 ceph-mon[35619]: Deploying daemon mds.g on smithi028 2024-09-19T09:30:26.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:25 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:boot 2024-09-19T09:30:26.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:25 smithi078 ceph-mon[35619]: fsmap 9 up:standby 2024-09-19T09:30:26.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:25 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.764 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:26 smithi028 systemd[1]: Starting Ceph mds.g for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:27.164 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:26 smithi028 ceph-mon[30972]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:27.164 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:26 smithi028 podman[71168]: 2024-09-19 09:30:26.782069642 +0000 UTC m=+0.016603411 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:27.164 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:26 smithi028 podman[71168]: 2024-09-19 09:30:26.902229654 +0000 UTC m=+0.136763452 container create deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T09:30:27.164 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 podman[71168]: 2024-09-19 09:30:27.163464655 +0000 UTC m=+0.397998420 container init deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T09:30:27.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:26 smithi038 ceph-mon[35271]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:27.179 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:27.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:26 smithi078 ceph-mon[35619]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:27.466 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:27.514 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 podman[71168]: 2024-09-19 09:30:27.166407446 +0000 UTC m=+0.400941200 container start deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T09:30:27.514 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 ceph-mds[71187]: -- 172.21.15.28:0/499486286 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e479539860 con 0x55e479517000 2024-09-19T09:30:27.514 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: starting mds.g at 2024-09-19T09:30:27.514 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 bash[71168]: deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 2024-09-19T09:30:27.514 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:30:27 smithi028 systemd[1]: Started Ceph mds.g for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 -- 172.21.15.28:0/2522384037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc108290 msgr2=0x7efebc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 --2- 172.21.15.28:0/2522384037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc108290 0x7efebc10ec30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009cd0 tx=0x7efeb402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 -- 172.21.15.28:0/2522384037 shutdown_connections 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 --2- 172.21.15.28:0/2522384037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc108290 0x7efebc10ec30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 --2- 172.21.15.28:0/2522384037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc103740 0x7efebc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 --2- 172.21.15.28:0/2522384037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc102d90 0x7efebc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 -- 172.21.15.28:0/2522384037 >> 172.21.15.28:0/2522384037 conn(0x7efebc0fc7d0 msgr2=0x7efebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 -- 172.21.15.28:0/2522384037 shutdown_connections 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.935+0000 7efec359a640 1 -- 172.21.15.28:0/2522384037 wait complete. 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 Processor -- start 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 -- start start 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103740 0x7efebc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc108290 0x7efebc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1120d0 con 0x7efebc102d90 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111f50 con 0x7efebc103740 2024-09-19T09:30:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec359a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efebc112250 con 0x7efebc108290 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec1336640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec1336640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49682/0 (socket says 172.21.15.28:49682) 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec1336640 1 -- 172.21.15.28:0/1905480409 learned_addr learned my addr 172.21.15.28:0/1905480409 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.936+0000 7efec0b35640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103740 0x7efebc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1b37640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc108290 0x7efebc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 -- 172.21.15.28:0/1905480409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc108290 msgr2=0x7efebc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc108290 0x7efebc19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 -- 172.21.15.28:0/1905480409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103740 msgr2=0x7efebc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103740 0x7efebc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 -- 172.21.15.28:0/1905480409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc19d560 con 0x7efebc102d90 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec1336640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d950 tx=0x7efeb800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8017070 con 0x7efebc102d90 2024-09-19T09:30:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb800de80 con 0x7efebc102d90 2024-09-19T09:30:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc19d850 con 0x7efebc102d90 2024-09-19T09:30:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f620 con 0x7efebc102d90 2024-09-19T09:30:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.937+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a5130 con 0x7efebc102d90 2024-09-19T09:30:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.939+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc103170 con 0x7efebc102d90 2024-09-19T09:30:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.940+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efeb8009c30 con 0x7efebc102d90 2024-09-19T09:30:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.941+0000 7efeb27fc640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 0x7efea80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.941+0000 7efec0b35640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 0x7efea80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.941+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7efeb80ceff0 con 0x7efebc102d90 2024-09-19T09:30:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.942+0000 7efec0b35640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 0x7efea80a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efebc0fee10 tx=0x7efeac009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:27.946+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efeb809c1b0 con 0x7efebc102d90 2024-09-19T09:30:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.166+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efebc10a2a0 con 0x7efebc102d90 2024-09-19T09:30:28.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.166+0000 7efeb27fc640 1 -- 172.21.15.28:0/1905480409 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8854 (secure 0 0 0) 0x7efeb809bab0 con 0x7efebc102d90 2024-09-19T09:30:28.168 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:28.168 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":11,"btime":"2024-09-19T09:30:27:916311+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T09:30:28.169 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 11 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 msgr2=0x7efea80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 0x7efea80a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efebc0fee10 tx=0x7efeac009110 comp rx=0 tx=0).stop 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 msgr2=0x7efebc1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d950 tx=0x7efeb800dd50 comp rx=0 tx=0).stop 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 shutdown_connections 2024-09-19T09:30:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efea80a32c0 0x7efea80a5780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc103740 0x7efebc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efebc108290 0x7efebc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 --2- 172.21.15.28:0/1905480409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efebc102d90 0x7efebc1985b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 >> 172.21.15.28:0/1905480409 conn(0x7efebc0fc7d0 msgr2=0x7efebc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.168+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 shutdown_connections 2024-09-19T09:30:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:28.169+0000 7efec359a640 1 -- 172.21.15.28:0/1905480409 wait complete. 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:28.501 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: Deploying daemon mds.a on smithi028 2024-09-19T09:30:28.502 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6838/1981224443,v1:172.21.15.28:6839/1981224443] up:boot 2024-09-19T09:30:28.502 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: fsmap 10 up:standby 2024-09-19T09:30:28.502 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:28.502 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:28 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1905480409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:28.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: Deploying daemon mds.a on smithi028 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6838/1981224443,v1:172.21.15.28:6839/1981224443] up:boot 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: fsmap 10 up:standby 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:28.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:28 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1905480409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:28.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:28.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:28.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: Deploying daemon mds.a on smithi028 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6838/1981224443,v1:172.21.15.28:6839/1981224443] up:boot 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: fsmap 10 up:standby 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:28.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:28 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1905480409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:29.605 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:29 smithi028 ceph-mon[30972]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:29 smithi078 ceph-mon[35619]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:29.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:29 smithi038 ceph-mon[35271]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:30.264 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 systemd[1]: Starting Ceph mds.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:30.571 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 podman[71587]: 2024-09-19 09:30:30.219820733 +0000 UTC m=+0.016417607 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:30.571 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 podman[71587]: 2024-09-19 09:30:30.323812763 +0000 UTC m=+0.120409638 container create 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:30.572 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 podman[71587]: 2024-09-19 09:30:30.542857613 +0000 UTC m=+0.339454495 container init 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:30.572 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 podman[71587]: 2024-09-19 09:30:30.5457444 +0000 UTC m=+0.342341281 container start 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T09:30:30.813 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:30.838 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 ceph-mds[71606]: -- 172.21.15.28:0/504922441 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562fea3a3c20 con 0x562fea381400 2024-09-19T09:30:30.838 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: starting mds.a at 2024-09-19T09:30:30.839 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 bash[71587]: 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 2024-09-19T09:30:30.839 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:30:30 smithi028 systemd[1]: Started Ceph mds.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:31.072 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:31.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:31.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:31 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- 172.21.15.28:0/3644998199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101068d0 msgr2=0x7fe410106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 --2- 172.21.15.28:0/3644998199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101068d0 0x7fe410106cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe400009930 tx=0x7fe40002a420 comp rx=0 tx=0).stop 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- 172.21.15.28:0/3644998199 shutdown_connections 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 --2- 172.21.15.28:0/3644998199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4101008d0 0x7fe410100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 --2- 172.21.15.28:0/3644998199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe410101270 0x7fe41010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 --2- 172.21.15.28:0/3644998199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101068d0 0x7fe410106cb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- 172.21.15.28:0/3644998199 >> 172.21.15.28:0/3644998199 conn(0x7fe4100fc610 msgr2=0x7fe4100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- 172.21.15.28:0/3644998199 shutdown_connections 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- 172.21.15.28:0/3644998199 wait complete. 2024-09-19T09:30:31.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 Processor -- start 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.514+0000 7fe40ffff640 1 -- start start 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe410101270 0x7fe4101987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4101068d0 0x7fe41019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110c90 con 0x7fe4101008d0 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110b10 con 0x7fe410101270 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40ffff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110e10 con 0x7fe4101068d0 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49712/0 (socket says 172.21.15.28:49712) 2024-09-19T09:30:31.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 -- 172.21.15.28:0/2194897748 learned_addr learned my addr 172.21.15.28:0/2194897748 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40f7fe640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4101068d0 0x7fe41019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe407fff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe410101270 0x7fe4101987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 -- 172.21.15.28:0/2194897748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4101068d0 msgr2=0x7fe41019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4101068d0 0x7fe41019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 -- 172.21.15.28:0/2194897748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe410101270 msgr2=0x7fe4101987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe410101270 0x7fe4101987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 -- 172.21.15.28:0/2194897748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41019d240 con 0x7fe4101008d0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.515+0000 7fe40effd640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe400009230 tx=0x7fe400004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.516+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40003f030 con 0x7fe4101008d0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.516+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe400003ad0 con 0x7fe4101008d0 2024-09-19T09:30:31.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.516+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41019d4d0 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.516+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035070 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.516+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a4e10 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.517+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe410100d30 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.518+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe400042030 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.519+0000 7fe40cff9640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 0x7fe3fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.519+0000 7fe407fff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 0x7fe3fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.519+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe4000efe80 con 0x7fe4101008d0 2024-09-19T09:30:31.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.519+0000 7fe407fff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 0x7fe3fc0a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4100ff080 tx=0x7fe3f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:31.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.522+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe4000bd040 con 0x7fe4101008d0 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:31.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:31 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:31.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:30:31.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:30:31.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:31 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:31.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.738+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4101995f0 con 0x7fe4101008d0 2024-09-19T09:30:31.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.738+0000 7fe40cff9640 1 -- 172.21.15.28:0/2194897748 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7fe4000bc940 con 0x7fe4101008d0 2024-09-19T09:30:31.741 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:31.741 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":12,"btime":"2024-09-19T09:30:31:329605+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T09:30:31.741 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 12 2024-09-19T09:30:31.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 msgr2=0x7fe3fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 0x7fe3fc0a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4100ff080 tx=0x7fe3f8005f90 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 msgr2=0x7fe410198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe400009230 tx=0x7fe400004270 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 shutdown_connections 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe3fc0a32c0 0x7fe3fc0a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe410101270 0x7fe4101987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4101068d0 0x7fe41019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 --2- 172.21.15.28:0/2194897748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4101008d0 0x7fe410198290 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 >> 172.21.15.28:0/2194897748 conn(0x7fe4100fc610 msgr2=0x7fe41006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:31.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 shutdown_connections 2024-09-19T09:30:31.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:31.741+0000 7fe40ffff640 1 -- 172.21.15.28:0/2194897748 wait complete. 2024-09-19T09:30:32.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: Deploying daemon mds.b on smithi038 2024-09-19T09:30:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6840/438294996,v1:172.21.15.28:6841/438294996] up:boot 2024-09-19T09:30:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: fsmap 11 up:standby 2024-09-19T09:30:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:32.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:32 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2194897748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:32.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: Deploying daemon mds.b on smithi038 2024-09-19T09:30:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6840/438294996,v1:172.21.15.28:6841/438294996] up:boot 2024-09-19T09:30:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: fsmap 11 up:standby 2024-09-19T09:30:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:32.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:32 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2194897748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: Deploying daemon mds.b on smithi038 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6840/438294996,v1:172.21.15.28:6841/438294996] up:boot 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: fsmap 11 up:standby 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:32 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2194897748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:32.837 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:32 smithi038 systemd[1]: Starting Ceph mds.b for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:30:33.139 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:32 smithi038 podman[65895]: 2024-09-19 09:30:32.835907897 +0000 UTC m=+0.108130456 container create e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:33.139 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:32 smithi038 podman[65895]: 2024-09-19 09:30:32.743380301 +0000 UTC m=+0.015602883 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:33.139 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 podman[65895]: 2024-09-19 09:30:33.046737053 +0000 UTC m=+0.318959618 container init e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T09:30:33.139 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 podman[65895]: 2024-09-19 09:30:33.049549849 +0000 UTC m=+0.321772412 container start e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T09:30:33.140 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 ceph-mds[65915]: -- 172.21.15.38:0/4010267999 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9f790b860 con 0x55d9f78e9000 2024-09-19T09:30:33.140 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: starting mds.b at 2024-09-19T09:30:33.140 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 bash[65895]: e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d 2024-09-19T09:30:33.425 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:30:33 smithi038 systemd[1]: Started Ceph mds.b for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:30:34.390 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:boot 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: fsmap 12 up:standby 2024-09-19T09:30:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:34 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:boot 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: fsmap 12 up:standby 2024-09-19T09:30:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:34 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:34.662 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:34.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:boot 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: fsmap 12 up:standby 2024-09-19T09:30:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:34 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:35.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 -- 172.21.15.28:0/4210903039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360101de0 msgr2=0x7fc3601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 --2- 172.21.15.28:0/4210903039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360101de0 0x7fc3601021c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc348009900 tx=0x7fc34802a410 comp rx=0 tx=0).stop 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 -- 172.21.15.28:0/4210903039 shutdown_connections 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 --2- 172.21.15.28:0/4210903039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360106f10 0x7fc36010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 --2- 172.21.15.28:0/4210903039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360102790 0x7fc3601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 --2- 172.21.15.28:0/4210903039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360101de0 0x7fc3601021c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 -- 172.21.15.28:0/4210903039 >> 172.21.15.28:0/4210903039 conn(0x7fc3600fd8c0 msgr2=0x7fc3600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 -- 172.21.15.28:0/4210903039 shutdown_connections 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.157+0000 7fc3657d7640 1 -- 172.21.15.28:0/4210903039 wait complete. 2024-09-19T09:30:35.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 Processor -- start 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 -- start start 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc360111f70 con 0x7fc360102790 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc360111df0 con 0x7fc360101de0 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc3657d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3601120f0 con 0x7fc360106f10 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35e7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35e7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43072/0 (socket says 172.21.15.28:43072) 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35e7fc640 1 -- 172.21.15.28:0/2633678923 learned_addr learned my addr 172.21.15.28:0/2633678923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:35.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35effd640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35f7fe640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35e7fc640 1 -- 172.21.15.28:0/2633678923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 msgr2=0x7fc360074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.158+0000 7fc35e7fc640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35e7fc640 1 -- 172.21.15.28:0/2633678923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 msgr2=0x7fc3600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35e7fc640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35e7fc640 1 -- 172.21.15.28:0/2633678923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3601a6910 con 0x7fc360102790 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35f7fe640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35effd640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc35e7fc640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d950 tx=0x7fc35000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350017070 con 0x7fc360102790 2024-09-19T09:30:35.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc350009c30 con 0x7fc360102790 2024-09-19T09:30:35.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3601a6c00 con 0x7fc360102790 2024-09-19T09:30:35.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000f6b0 con 0x7fc360102790 2024-09-19T09:30:35.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.159+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3601a7190 con 0x7fc360102790 2024-09-19T09:30:35.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.160+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3601021c0 con 0x7fc360102790 2024-09-19T09:30:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.161+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc3500109d0 con 0x7fc360102790 2024-09-19T09:30:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.162+0000 7fc357fff640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 0x7fc34c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.162+0000 7fc35effd640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 0x7fc34c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.162+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc3500cf000 con 0x7fc360102790 2024-09-19T09:30:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.162+0000 7fc35effd640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 0x7fc34c0a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc348009230 tx=0x7fc3480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:35.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.166+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc350009de0 con 0x7fc360102790 2024-09-19T09:30:35.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.382+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc36010a0f0 con 0x7fc360102790 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.385 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.385 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:35.385 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:35.385 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:35 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.382+0000 7fc357fff640 1 -- 172.21.15.28:0/2633678923 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7fc35009c1c0 con 0x7fc360102790 2024-09-19T09:30:35.386 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:35.386 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":13,"btime":"2024-09-19T09:30:33:346697+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T09:30:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 13 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 msgr2=0x7fc34c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 0x7fc34c0a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc348009230 tx=0x7fc3480058d0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 msgr2=0x7fc360074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d950 tx=0x7fc35000de10 comp rx=0 tx=0).stop 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 shutdown_connections 2024-09-19T09:30:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc34c0a32c0 0x7fc34c0a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 --2- 172.21.15.28:0/2633678923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 >> 172.21.15.28:0/2633678923 conn(0x7fc3600fd8c0 msgr2=0x7fc3600fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 shutdown_connections 2024-09-19T09:30:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:35.385+0000 7fc3657d7640 1 -- 172.21.15.28:0/2633678923 wait complete. 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:35.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:35 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:35 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:36.015 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T09:30:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:36 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:36 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2633678923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:36 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:36 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2633678923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:36 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:36 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2633678923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:36.919 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:37.301 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:37 smithi028 ceph-mon[30972]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:37.302 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.302 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.302 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.302 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:37 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 -- 172.21.15.28:0/3692170172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 msgr2=0x7f85481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 --2- 172.21.15.28:0/3692170172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481067e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f854402a410 comp rx=0 tx=0).stop 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 -- 172.21.15.28:0/3692170172 shutdown_connections 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 --2- 172.21.15.28:0/3692170172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548106f10 0x7f854810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 --2- 172.21.15.28:0/3692170172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481067e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 --2- 172.21.15.28:0/3692170172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548101de0 0x7f85481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 -- 172.21.15.28:0/3692170172 >> 172.21.15.28:0/3692170172 conn(0x7f85480fd8c0 msgr2=0x7f85480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 -- 172.21.15.28:0/3692170172 shutdown_connections 2024-09-19T09:30:37.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 -- 172.21.15.28:0/3692170172 wait complete. 2024-09-19T09:30:37.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.315+0000 7f854ff85640 1 Processor -- start 2024-09-19T09:30:37.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 -- start start 2024-09-19T09:30:37.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548101de0 0x7f8548198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:37.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548106f10 0x7f854819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8548111f70 con 0x7f8548102790 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8548111df0 con 0x7f8548106f10 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854ff85640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85481120f0 con 0x7f8548101de0 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43086/0 (socket says 172.21.15.28:43086) 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 -- 172.21.15.28:0/694208661 learned_addr learned my addr 172.21.15.28:0/694208661 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854dd21640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548101de0 0x7f8548198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:37.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854e522640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548106f10 0x7f854819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 -- 172.21.15.28:0/694208661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548101de0 msgr2=0x7f8548198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548101de0 0x7f8548198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 -- 172.21.15.28:0/694208661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548106f10 msgr2=0x7f854819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548106f10 0x7f854819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 -- 172.21.15.28:0/694208661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854819d440 con 0x7f8548102790 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.316+0000 7f854d520640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8544006010 tx=0x7f85440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.317+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854403f030 con 0x7f8548102790 2024-09-19T09:30:37.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.317+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544003ce0 con 0x7f8548102790 2024-09-19T09:30:37.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.317+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854819d6d0 con 0x7f8548102790 2024-09-19T09:30:37.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.317+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544035070 con 0x7f8548102790 2024-09-19T09:30:37.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.317+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85481a5010 con 0x7f8548102790 2024-09-19T09:30:37.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.318+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85481032c0 con 0x7f8548102790 2024-09-19T09:30:37.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.319+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8544042030 con 0x7f8548102790 2024-09-19T09:30:37.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.320+0000 7f853effd640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 0x7f85340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:37.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.320+0000 7f854dd21640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 0x7f85340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:37.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.320+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f85440ef5a0 con 0x7f8548102790 2024-09-19T09:30:37.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.320+0000 7f854dd21640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 0x7f85340a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8538005d00 tx=0x7f853800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:37.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.324+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85440bc1d0 con 0x7f8548102790 2024-09-19T09:30:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.514+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f854810a0f0 con 0x7f8548102790 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.515+0000 7f853effd640 1 -- 172.21.15.28:0/694208661 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f85440c04d0 con 0x7f8548102790 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-19T09:30:37.517 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-19T09:30:37.518 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:30:37.518 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-19T09:30:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 msgr2=0x7f85340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 0x7f85340a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8538005d00 tx=0x7f853800b410 comp rx=0 tx=0).stop 2024-09-19T09:30:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 msgr2=0x7f85481989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8544006010 tx=0x7f85440042f0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 shutdown_connections 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f85340a3240 0x7f85340a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8548106f10 0x7f854819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.517+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8548102790 0x7f85481989d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.518+0000 7f854ff85640 1 --2- 172.21.15.28:0/694208661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8548101de0 0x7f8548198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.518+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 >> 172.21.15.28:0/694208661 conn(0x7f85480fd8c0 msgr2=0x7f85480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.518+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 shutdown_connections 2024-09-19T09:30:37.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:37.518+0000 7f854ff85640 1 -- 172.21.15.28:0/694208661 wait complete. 2024-09-19T09:30:37.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:37 smithi078 ceph-mon[35619]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:37.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:37 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:37 smithi038 ceph-mon[35271]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:37 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.141 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T09:30:38.141 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T09:30:38.403 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/694208661' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:38.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/694208661' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 -- 172.21.15.28:0/1900483068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc278102b90 msgr2=0x7fc278102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 --2- 172.21.15.28:0/1900483068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc278102b90 0x7fc278102f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc274009900 tx=0x7fc27402a480 comp rx=0 tx=0).stop 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 -- 172.21.15.28:0/1900483068 shutdown_connections 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 --2- 172.21.15.28:0/1900483068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc27810bfc0 0x7fc27810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 --2- 172.21.15.28:0/1900483068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 0x7fc27810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:38.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 --2- 172.21.15.28:0/1900483068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc278102b90 0x7fc278102f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:38.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 -- 172.21.15.28:0/1900483068 >> 172.21.15.28:0/1900483068 conn(0x7fc2780fc630 msgr2=0x7fc2780fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:38.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 -- 172.21.15.28:0/1900483068 shutdown_connections 2024-09-19T09:30:38.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.843+0000 7fc27e39e640 1 -- 172.21.15.28:0/1900483068 wait complete. 2024-09-19T09:30:38.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 Processor -- start 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 -- start start 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278102b90 0x7fc2781982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 0x7fc278198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc278111dc0 con 0x7fc27810bfc0 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc278111c40 con 0x7fc278102b90 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27e39e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc278111f40 con 0x7fc2781034b0 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27db9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27db9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43114/0 (socket says 172.21.15.28:43114) 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27db9d640 1 -- 172.21.15.28:0/882904826 learned_addr learned my addr 172.21.15.28:0/882904826 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:38.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27cb9b640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 0x7fc278198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.844+0000 7fc27d39c640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278102b90 0x7fc2781982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 -- 172.21.15.28:0/882904826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 msgr2=0x7fc278198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 0x7fc278198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 -- 172.21.15.28:0/882904826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278102b90 msgr2=0x7fc2781982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278102b90 0x7fc2781982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 -- 172.21.15.28:0/882904826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27819d290 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27db9d640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc27000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270017070 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc270003e50 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2700106c0 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27819d580 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.845+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2781a4e60 con 0x7fc27810bfc0 2024-09-19T09:30:38.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.846+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc278102f70 con 0x7fc27810bfc0 2024-09-19T09:30:38.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.848+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc270003970 con 0x7fc27810bfc0 2024-09-19T09:30:38.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.849+0000 7fc26e7fc640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 0x7fc2640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:38.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.849+0000 7fc27d39c640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 0x7fc2640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:38.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.849+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc2700cf180 con 0x7fc27810bfc0 2024-09-19T09:30:38.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.849+0000 7fc27d39c640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 0x7fc2640a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc274005ab0 tx=0x7fc274005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:38.852+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc27009c320 con 0x7fc27810bfc0 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/694208661' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:30:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:30:39.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.031+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc278199640 con 0x7fc27810bfc0 2024-09-19T09:30:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.516+0000 7fc26e7fc640 1 -- 172.21.15.28:0/882904826 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fc278199640 con 0x7fc27810bfc0 2024-09-19T09:30:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-19T09:30:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 msgr2=0x7fc2640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 0x7fc2640a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc274005ab0 tx=0x7fc274005a20 comp rx=0 tx=0).stop 2024-09-19T09:30:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 msgr2=0x7fc27819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc27000b260 comp rx=0 tx=0).stop 2024-09-19T09:30:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 shutdown_connections 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc2640a32c0 0x7fc2640a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc27810bfc0 0x7fc27819cbb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2781034b0 0x7fc278198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 --2- 172.21.15.28:0/882904826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278102b90 0x7fc2781982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 >> 172.21.15.28:0/882904826 conn(0x7fc2780fc630 msgr2=0x7fc2780fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 shutdown_connections 2024-09-19T09:30:39.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:39.518+0000 7fc27e39e640 1 -- 172.21.15.28:0/882904826 wait complete. 2024-09-19T09:30:39.820 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:39 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:30:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:39 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:30:39.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:39 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:30:40.124 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T09:30:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:40.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:40 smithi028 ceph-mon[30972]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:40.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:40 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:30:40.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:40 smithi028 ceph-mon[30972]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:30:40.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:40 smithi028 ceph-mon[30972]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:30:40.799 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:40 smithi078 ceph-mon[35619]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:40.799 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:40 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:30:40.799 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:40 smithi078 ceph-mon[35619]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:30:40.799 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:40 smithi078 ceph-mon[35619]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:30:40.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.813+0000 7fd7b430e640 1 -- 172.21.15.28:0/1549371139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 msgr2=0x7fd7ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.813+0000 7fd7b430e640 1 --2- 172.21.15.28:0/1549371139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac1029b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c02a480 comp rx=0 tx=0).stop 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- 172.21.15.28:0/1549371139 shutdown_connections 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- 172.21.15.28:0/1549371139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac102ef0 0x7fd7ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- 172.21.15.28:0/1549371139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac1029b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- 172.21.15.28:0/1549371139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac100e20 0x7fd7ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- 172.21.15.28:0/1549371139 >> 172.21.15.28:0/1549371139 conn(0x7fd7ac0fc7d0 msgr2=0x7fd7ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- 172.21.15.28:0/1549371139 shutdown_connections 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- 172.21.15.28:0/1549371139 wait complete. 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 Processor -- start 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- start start 2024-09-19T09:30:40.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 0x7fd7ac1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac102ef0 0x7fd7ac1a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac110fa0 con 0x7fd7ac102570 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac110e20 con 0x7fd7ac102ef0 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.814+0000 7fd7b430e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac111120 con 0x7fd7ac100e20 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43138/0 (socket says 172.21.15.28:43138) 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 -- 172.21.15.28:0/847275418 learned_addr learned my addr 172.21.15.28:0/847275418 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b28ab640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac102ef0 0x7fd7ac1a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b20aa640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 0x7fd7ac1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 -- 172.21.15.28:0/847275418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 msgr2=0x7fd7ac1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:40.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 0x7fd7ac1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 -- 172.21.15.28:0/847275418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac102ef0 msgr2=0x7fd7ac1a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac102ef0 0x7fd7ac1a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 -- 172.21.15.28:0/847275418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac1a49b0 con 0x7fd7ac102570 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b20aa640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 0x7fd7ac1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b18a9640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006010 tx=0x7fd79c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c03f070 con 0x7fd7ac102570 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c031c30 con 0x7fd7ac102570 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac1a4b80 con 0x7fd7ac102570 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c035030 con 0x7fd7ac102570 2024-09-19T09:30:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.815+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac1a5030 con 0x7fd7ac102570 2024-09-19T09:30:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.816+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac101200 con 0x7fd7ac102570 2024-09-19T09:30:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.818+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd79c042070 con 0x7fd7ac102570 2024-09-19T09:30:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.818+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 0x7fd7980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.818+0000 7fd7b20aa640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 0x7fd7980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:40.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.818+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fd79c0f02a0 con 0x7fd7ac102570 2024-09-19T09:30:40.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.819+0000 7fd7b20aa640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 0x7fd7980a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80058c0 tx=0x7fd7a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:40.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.821+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd79c031de0 con 0x7fd7ac102570 2024-09-19T09:30:40.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:40 smithi038 ceph-mon[35271]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:40.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:40 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/882904826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:30:40.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:40 smithi038 ceph-mon[35271]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:30:40.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:40 smithi038 ceph-mon[35271]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:30:40.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:40.996+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd7ac109090 con 0x7fd7ac102570 2024-09-19T09:30:41.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.529+0000 7fd7a37fe640 1 -- 172.21.15.28:0/847275418 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fd79c0bd3b0 con 0x7fd7ac102570 2024-09-19T09:30:41.531 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 msgr2=0x7fd7980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 0x7fd7980a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80058c0 tx=0x7fd7a8005850 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 msgr2=0x7fd7ac199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006010 tx=0x7fd79c003c10 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 shutdown_connections 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7980a3240 0x7fd7980a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7ac102ef0 0x7fd7ac1a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7ac102570 0x7fd7ac199930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 --2- 172.21.15.28:0/847275418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7ac100e20 0x7fd7ac1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:41.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 >> 172.21.15.28:0/847275418 conn(0x7fd7ac0fc7d0 msgr2=0x7fd7ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:41.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 shutdown_connections 2024-09-19T09:30:41.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:41.532+0000 7fd7b430e640 1 -- 172.21.15.28:0/847275418 wait complete. 2024-09-19T09:30:41.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:41 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:30:41.886 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:41 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:30:41.930 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:41 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:30:42.156 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T09:30:42.410 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:42.831 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:42 smithi028 ceph-mon[30972]: pgmap v274: 33 pgs: 3 creating+peering, 22 unknown, 8 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:42.832 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:42 smithi028 ceph-mon[30972]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:30:42.832 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:42 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:30:42.832 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:42 smithi028 ceph-mon[30972]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 -- 172.21.15.28:0/4224842497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 msgr2=0x7f0d64110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/4224842497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d64110870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009ca0 tx=0x7f0d5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 -- 172.21.15.28:0/4224842497 shutdown_connections 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/4224842497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 0x7f0d6410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/4224842497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d64110870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/4224842497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 0x7f0d64100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 -- 172.21.15.28:0/4224842497 >> 172.21.15.28:0/4224842497 conn(0x7f0d640fc7a0 msgr2=0x7f0d640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:42.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 -- 172.21.15.28:0/4224842497 shutdown_connections 2024-09-19T09:30:42.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.832+0000 7f0d6bb92640 1 -- 172.21.15.28:0/4224842497 wait complete. 2024-09-19T09:30:42.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 Processor -- start 2024-09-19T09:30:42.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 -- start start 2024-09-19T09:30:42.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 0x7f0d6419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:42.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 0x7f0d6419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64114220 con 0x7f0d6410e480 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641140a0 con 0x7f0d64101280 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6bb92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d641143a0 con 0x7f0d641008d0 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6a12f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6a12f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43166/0 (socket says 172.21.15.28:43166) 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6a12f640 1 -- 172.21.15.28:0/2933640764 learned_addr learned my addr 172.21.15.28:0/2933640764 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.833+0000 7f0d6912d640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 0x7f0d6419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6992e640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 0x7f0d6419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 -- 172.21.15.28:0/2933640764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 msgr2=0x7f0d6419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 0x7f0d6419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 -- 172.21.15.28:0/2933640764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 msgr2=0x7f0d6419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 0x7f0d6419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 -- 172.21.15.28:0/2933640764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6419f670 con 0x7f0d6410e480 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6a12f640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c005e30 tx=0x7f0d5c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c031650 con 0x7f0d6410e480 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5c0317f0 con 0x7f0d6410e480 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6419f900 con 0x7f0d6410e480 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c02fa30 con 0x7f0d6410e480 2024-09-19T09:30:42.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.834+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d641a7240 con 0x7f0d6410e480 2024-09-19T09:30:42.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.835+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d64100cb0 con 0x7f0d6410e480 2024-09-19T09:30:42.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.836+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0d5c02fbd0 con 0x7f0d6410e480 2024-09-19T09:30:42.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.837+0000 7f0d5affd640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 0x7f0d500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:42.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.837+0000 7f0d6992e640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 0x7f0d500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:42.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.837+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f0d5c0f0aa0 con 0x7f0d6410e480 2024-09-19T09:30:42.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.838+0000 7f0d6992e640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 0x7f0d500a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009990 tx=0x7f0d60009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:42.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:42.842+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d5c0bd970 con 0x7f0d6410e480 2024-09-19T09:30:42.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:42 smithi078 ceph-mon[35619]: pgmap v274: 33 pgs: 3 creating+peering, 22 unknown, 8 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:42 smithi078 ceph-mon[35619]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:42 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:42 smithi078 ceph-mon[35619]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:30:42.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:42 smithi038 ceph-mon[35271]: pgmap v274: 33 pgs: 3 creating+peering, 22 unknown, 8 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:42.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:42 smithi038 ceph-mon[35271]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:30:42.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:42 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/847275418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:30:42.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:42 smithi038 ceph-mon[35271]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:30:43.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.053+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0d64109080 con 0x7f0d6410e480 2024-09-19T09:30:43.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.551+0000 7f0d5affd640 1 -- 172.21.15.28:0/2933640764 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f0d5c0319a0 con 0x7f0d6410e480 2024-09-19T09:30:43.552 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 msgr2=0x7f0d500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 0x7f0d500a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009990 tx=0x7f0d60009110 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 msgr2=0x7f0d6419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c005e30 tx=0x7f0d5c003f10 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 shutdown_connections 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.554+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d500a32c0 0x7f0d500a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d64101280 0x7f0d6419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6410e480 0x7f0d6419ef90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 --2- 172.21.15.28:0/2933640764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d641008d0 0x7f0d6419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 >> 172.21.15.28:0/2933640764 conn(0x7f0d640fc7a0 msgr2=0x7f0d640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 shutdown_connections 2024-09-19T09:30:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:43.555+0000 7f0d6bb92640 1 -- 172.21.15.28:0/2933640764 wait complete. 2024-09-19T09:30:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:43 smithi078 ceph-mon[35619]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:30:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:43 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:30:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:43 smithi078 ceph-mon[35619]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T09:30:43.867 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:43 smithi028 ceph-mon[30972]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:30:43.868 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:43 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:30:43.868 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:43 smithi028 ceph-mon[30972]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T09:30:43.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:43 smithi038 ceph-mon[35271]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:30:43.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:43 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:30:43.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:43 smithi038 ceph-mon[35271]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T09:30:44.137 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:30:44.400 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:44.813 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:44 smithi028 ceph-mon[30972]: pgmap v277: 97 pgs: 17 creating+peering, 38 unknown, 42 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:44.813 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:44 smithi028 ceph-mon[30972]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:30:44.813 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:44 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:30:44.813 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:44 smithi028 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T09:30:44.813 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:44 smithi028 ceph-mon[30972]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T09:30:44.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.835+0000 7fd9716cc640 1 -- 172.21.15.28:0/1500464875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102ef0 msgr2=0x7fd96c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:44.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.835+0000 7fd9716cc640 1 --2- 172.21.15.28:0/1500464875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102ef0 0x7fd96c10da40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd964009900 tx=0x7fd96402a480 comp rx=0 tx=0).stop 2024-09-19T09:30:44.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.835+0000 7fd9716cc640 1 -- 172.21.15.28:0/1500464875 shutdown_connections 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.835+0000 7fd9716cc640 1 --2- 172.21.15.28:0/1500464875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102ef0 0x7fd96c10da40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 --2- 172.21.15.28:0/1500464875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c102570 0x7fd96c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 --2- 172.21.15.28:0/1500464875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c100e20 0x7fd96c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 -- 172.21.15.28:0/1500464875 >> 172.21.15.28:0/1500464875 conn(0x7fd96c0fc7d0 msgr2=0x7fd96c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 -- 172.21.15.28:0/1500464875 shutdown_connections 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 -- 172.21.15.28:0/1500464875 wait complete. 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 Processor -- start 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.836+0000 7fd9716cc640 1 -- start start 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 0x7fd96c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 0x7fd96c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1120a0 con 0x7fd96c102570 2024-09-19T09:30:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c111f20 con 0x7fd96c102ef0 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c112220 con 0x7fd96c100e20 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49688/0 (socket says 172.21.15.28:49688) 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 -- 172.21.15.28:0/2110172036 learned_addr learned my addr 172.21.15.28:0/2110172036 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96b7fe640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 0x7fd96c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96affd640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 0x7fd96c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 -- 172.21.15.28:0/2110172036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 msgr2=0x7fd96c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 0x7fd96c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 -- 172.21.15.28:0/2110172036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 msgr2=0x7fd96c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 0x7fd96c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 -- 172.21.15.28:0/2110172036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c19d450 con 0x7fd96c102570 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96affd640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 0x7fd96c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:44.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96b7fe640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 0x7fd96c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd96a7fc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d770 tx=0x7fd96000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960017070 con 0x7fd96c102570 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd960005500 con 0x7fd96c102570 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c19d740 con 0x7fd96c102570 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.837+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960004e00 con 0x7fd96c102570 2024-09-19T09:30:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.838+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c1a5020 con 0x7fd96c102570 2024-09-19T09:30:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.839+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c101200 con 0x7fd96c102570 2024-09-19T09:30:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.840+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd960004fa0 con 0x7fd96c102570 2024-09-19T09:30:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.840+0000 7fd953fff640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 0x7fd9540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.841+0000 7fd96affd640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 0x7fd9540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.841+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd9600cf2f0 con 0x7fd96c102570 2024-09-19T09:30:44.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.841+0000 7fd96affd640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 0x7fd9540a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005fd0 tx=0x7fd95c0093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:44.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:44.845+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd96009c160 con 0x7fd96c102570 2024-09-19T09:30:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-mon[35619]: pgmap v277: 97 pgs: 17 creating+peering, 38 unknown, 42 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-mon[35619]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:30:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:30:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-mon[35619]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T09:30:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-mon[35619]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T09:30:44.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:44 smithi038 ceph-mon[35271]: pgmap v277: 97 pgs: 17 creating+peering, 38 unknown, 42 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:44 smithi038 ceph-mon[35271]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:30:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:44 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2933640764' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:30:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:44 smithi038 ceph-mon[35271]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T09:30:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:44 smithi038 ceph-mon[35271]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T09:30:45.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.021+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd96c1a51f0 con 0x7fd96c102570 2024-09-19T09:30:45.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.022+0000 7fd953fff640 1 -- 172.21.15.28:0/2110172036 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22195 (secure 0 0 0) 0x7fd96009ba60 con 0x7fd96c102570 2024-09-19T09:30:45.024 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:45.024 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":80,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:30:43.054505+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 msgr2=0x7fd9540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 0x7fd9540a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005fd0 tx=0x7fd95c0093e0 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 msgr2=0x7fd96c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d770 tx=0x7fd96000dc30 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 shutdown_connections 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd9540a3240 0x7fd9540a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd96c102ef0 0x7fd96c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd96c102570 0x7fd96c1989e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 --2- 172.21.15.28:0/2110172036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd96c100e20 0x7fd96c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 >> 172.21.15.28:0/2110172036 conn(0x7fd96c0fc7d0 msgr2=0x7fd96c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 shutdown_connections 2024-09-19T09:30:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:45.025+0000 7fd9716cc640 1 -- 172.21.15.28:0/2110172036 wait complete. 2024-09-19T09:30:45.357 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:30:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: 2024-09-19T09:30:44.906+0000 7fb419b2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T09:30:45.693 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:30:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:active 2024-09-19T09:30:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:45 smithi078 ceph-mon[35619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:45 smithi078 ceph-mon[35619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:45 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2110172036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:45.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:active 2024-09-19T09:30:45.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:45 smithi038 ceph-mon[35271]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:45.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:45 smithi038 ceph-mon[35271]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:45.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:45 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2110172036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:45.952 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] up:active 2024-09-19T09:30:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:45 smithi028 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:45 smithi028 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:46.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:45 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2110172036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:46.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 -- 172.21.15.28:0/310077086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610103720 msgr2=0x7f361010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:46.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 --2- 172.21.15.28:0/310077086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610103720 0x7f361010bc10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009900 tx=0x7f35fc02a480 comp rx=0 tx=0).stop 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 -- 172.21.15.28:0/310077086 shutdown_connections 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 --2- 172.21.15.28:0/310077086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361010c1a0 0x7f361010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 --2- 172.21.15.28:0/310077086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610103720 0x7f361010bc10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 --2- 172.21.15.28:0/310077086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610102e00 0x7f36101031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 -- 172.21.15.28:0/310077086 >> 172.21.15.28:0/310077086 conn(0x7f36100fc7a0 msgr2=0x7f36100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 -- 172.21.15.28:0/310077086 shutdown_connections 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.394+0000 7f3617008640 1 -- 172.21.15.28:0/310077086 wait complete. 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 Processor -- start 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 -- start start 2024-09-19T09:30:46.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 0x7f3610100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361010c1a0 0x7f36101014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3610111fb0 con 0x7f3610102e00 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3610111e30 con 0x7f3610103720 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3617008640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3610112130 con 0x7f361010c1a0 2024-09-19T09:30:46.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.395+0000 7f3614da4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49714/0 (socket says 172.21.15.28:49714) 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 -- 172.21.15.28:0/3638482444 learned_addr learned my addr 172.21.15.28:0/3638482444 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f360ffff640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 0x7f3610100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 -- 172.21.15.28:0/3638482444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361010c1a0 msgr2=0x7f36101014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361010c1a0 0x7f36101014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 -- 172.21.15.28:0/3638482444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 msgr2=0x7f3610100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 0x7f3610100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 -- 172.21.15.28:0/3638482444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36101a68d0 con 0x7f3610102e00 2024-09-19T09:30:46.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f360ffff640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 0x7f3610100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3614da4640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f360400c920 tx=0x7f360400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604007c70 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3604007e10 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36101a6bc0 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604018890 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.396+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36101a7100 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.397+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36101031e0 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.398+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3604018a70 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.399+0000 7f360dffb640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 0x7f36000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.399+0000 7f360ffff640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 0x7f36000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.399+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f36040cf9c0 con 0x7f3610102e00 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.400+0000 7f360ffff640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 0x7f36000a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:46.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.403+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f36040d1e20 con 0x7f3610102e00 2024-09-19T09:30:46.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.582+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3610101f90 con 0x7f3610102e00 2024-09-19T09:30:46.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.583+0000 7f360dffb640 1 -- 172.21.15.28:0/3638482444 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22195 (secure 0 0 0) 0x7f3604014030 con 0x7f3610102e00 2024-09-19T09:30:46.585 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:30:46.585 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":80,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:30:43.054505+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 msgr2=0x7f36000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 0x7f36000a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc032d50 comp rx=0 tx=0).stop 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 msgr2=0x7f3610100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f360400c920 tx=0x7f360400cde0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 shutdown_connections 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36000a3240 0x7f36000a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3610103720 0x7f3610100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361010c1a0 0x7f36101014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 --2- 172.21.15.28:0/3638482444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3610102e00 0x7f3610100a60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:46.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 >> 172.21.15.28:0/3638482444 conn(0x7f36100fc7a0 msgr2=0x7f36100fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:46.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 shutdown_connections 2024-09-19T09:30:46.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:46.586+0000 7f3617008640 1 -- 172.21.15.28:0/3638482444 wait complete. 2024-09-19T09:30:46.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:46 smithi028 ceph-mon[30972]: pgmap v279: 97 pgs: 14 creating+peering, 13 unknown, 70 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T09:30:46.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:46 smithi028 ceph-mon[30972]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:30:46.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:46 smithi028 ceph-mon[30972]: Cluster is now healthy 2024-09-19T09:30:46.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:46 smithi078 ceph-mon[35619]: pgmap v279: 97 pgs: 14 creating+peering, 13 unknown, 70 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T09:30:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:46 smithi078 ceph-mon[35619]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:30:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:46 smithi078 ceph-mon[35619]: Cluster is now healthy 2024-09-19T09:30:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:46 smithi038 ceph-mon[35271]: pgmap v279: 97 pgs: 14 creating+peering, 13 unknown, 70 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T09:30:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:46 smithi038 ceph-mon[35271]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:30:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:46 smithi038 ceph-mon[35271]: Cluster is now healthy 2024-09-19T09:30:47.175 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T09:30:47.433 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:47.805 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:47 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3638482444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:47 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3638482444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:47.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:47 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3638482444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 -- 172.21.15.28:0/4138366777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44102790 msgr2=0x7faf441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 --2- 172.21.15.28:0/4138366777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44102790 0x7faf441067e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009900 tx=0x7faf2c02a410 comp rx=0 tx=0).stop 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 -- 172.21.15.28:0/4138366777 shutdown_connections 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 --2- 172.21.15.28:0/4138366777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44106f10 0x7faf4410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 --2- 172.21.15.28:0/4138366777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44102790 0x7faf441067e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 --2- 172.21.15.28:0/4138366777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44101de0 0x7faf441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 -- 172.21.15.28:0/4138366777 >> 172.21.15.28:0/4138366777 conn(0x7faf440fd8c0 msgr2=0x7faf440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.987+0000 7faf49b39640 1 -- 172.21.15.28:0/4138366777 shutdown_connections 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.988+0000 7faf49b39640 1 -- 172.21.15.28:0/4138366777 wait complete. 2024-09-19T09:30:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.988+0000 7faf49b39640 1 Processor -- start 2024-09-19T09:30:47.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.988+0000 7faf49b39640 1 -- start start 2024-09-19T09:30:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 0x7faf441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 0x7faf4419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faf44111f90 con 0x7faf44101de0 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faf44111e10 con 0x7faf44106f10 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf44112110 con 0x7faf44102790 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49742/0 (socket says 172.21.15.28:49742) 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 -- 172.21.15.28:0/3354855638 learned_addr learned my addr 172.21.15.28:0/3354855638 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf43fff640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 0x7faf4419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf3affd640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 0x7faf441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 -- 172.21.15.28:0/3354855638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 msgr2=0x7faf441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 0x7faf441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 -- 172.21.15.28:0/3354855638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 msgr2=0x7faf4419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 0x7faf4419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 -- 172.21.15.28:0/3354855638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4419d420 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf3affd640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 0x7faf441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf43fff640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 0x7faf4419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf437fe640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c890 tx=0x7faf3400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34004220 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf340043c0 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.989+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4419d710 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.990+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340049e0 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.990+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441a4ff0 con 0x7faf44101de0 2024-09-19T09:30:47.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.991+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf441021c0 con 0x7faf44101de0 2024-09-19T09:30:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.992+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7faf340079b0 con 0x7faf44101de0 2024-09-19T09:30:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.993+0000 7faf417fa640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 0x7faf300a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.993+0000 7faf3affd640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 0x7faf300a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.993+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7faf340cf890 con 0x7faf44101de0 2024-09-19T09:30:47.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.993+0000 7faf3affd640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 0x7faf300a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf2c000c00 tx=0x7faf2c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:47.997+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf34014030 con 0x7faf44101de0 2024-09-19T09:30:48.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.209+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf4402d7c0 con 0x7faf44101de0 2024-09-19T09:30:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.584+0000 7faf417fa640 1 -- 172.21.15.28:0/3354855638 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7faf3409c700 con 0x7faf44101de0 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 msgr2=0x7faf300a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 0x7faf300a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf2c000c00 tx=0x7faf2c032d50 comp rx=0 tx=0).stop 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 msgr2=0x7faf44198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c890 tx=0x7faf3400cd50 comp rx=0 tx=0).stop 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 shutdown_connections 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faf300a32c0 0x7faf300a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faf44106f10 0x7faf4419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf44102790 0x7faf441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:48.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 --2- 172.21.15.28:0/3354855638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf44101de0 0x7faf44198470 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:48.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 >> 172.21.15.28:0/3354855638 conn(0x7faf440fd8c0 msgr2=0x7faf440fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:48.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 shutdown_connections 2024-09-19T09:30:48.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:48.586+0000 7faf49b39640 1 -- 172.21.15.28:0/3354855638 wait complete. 2024-09-19T09:30:48.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:48 smithi078 ceph-mon[35619]: pgmap v280: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-19T09:30:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:30:48.889 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:48 smithi028 ceph-mon[30972]: pgmap v280: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-19T09:30:48.889 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:30:48.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:48 smithi038 ceph-mon[35271]: pgmap v280: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-19T09:30:48.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:30:49.167 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T09:30:49.167 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T09:30:49.428 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:49 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:30:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:49 smithi078 ceph-mon[35619]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:49.869 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:49 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:30:49.869 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:49 smithi028 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 -- 172.21.15.28:0/3428676652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880106f10 msgr2=0x7f688010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 --2- 172.21.15.28:0/3428676652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880106f10 0x7f688010ea90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6878009ca0 tx=0x7f687802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 -- 172.21.15.28:0/3428676652 shutdown_connections 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 --2- 172.21.15.28:0/3428676652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880106f10 0x7f688010ea90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 --2- 172.21.15.28:0/3428676652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880102790 0x7f68801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 --2- 172.21.15.28:0/3428676652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 0x7f68801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.867+0000 7f6887a33640 1 -- 172.21.15.28:0/3428676652 >> 172.21.15.28:0/3428676652 conn(0x7f68800fd8c0 msgr2=0x7f68800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.868+0000 7f6887a33640 1 -- 172.21.15.28:0/3428676652 shutdown_connections 2024-09-19T09:30:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.868+0000 7f6887a33640 1 -- 172.21.15.28:0/3428676652 wait complete. 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 Processor -- start 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 -- start start 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111f90 con 0x7f6880102790 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111e10 con 0x7f6880101de0 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6887a33640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6880112110 con 0x7f6880106f10 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49758/0 (socket says 172.21.15.28:49758) 2024-09-19T09:30:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 -- 172.21.15.28:0/162726351 learned_addr learned my addr 172.21.15.28:0/162726351 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6885fd0640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f68857cf640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 -- 172.21.15.28:0/162726351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880106f10 msgr2=0x7f6880074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 -- 172.21.15.28:0/162726351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 msgr2=0x7f68800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.869+0000 7f6884fce640 1 -- 172.21.15.28:0/162726351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68801a69c0 con 0x7f6880102790 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f6884fce640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f687000b4a0 tx=0x7f687000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870014070 con 0x7f6880102790 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68801a6cb0 con 0x7f6880102790 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870010af0 con 0x7f6880102790 2024-09-19T09:30:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870017d10 con 0x7f6880102790 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.870+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a71a0 con 0x7f6880102790 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.871+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68801021c0 con 0x7f6880102790 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.872+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6870017070 con 0x7f6880102790 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.873+0000 7f68767fc640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 0x7f686c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.873+0000 7f68857cf640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 0x7f686c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.873+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f68700ceb90 con 0x7f6880102790 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.873+0000 7f68857cf640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 0x7f686c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f687c005fd0 tx=0x7f687c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:49.877+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f687009ba00 con 0x7f6880102790 2024-09-19T09:30:49.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:49 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3354855638' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:30:49.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:49 smithi038 ceph-mon[35271]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T09:30:50.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.089+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f688010a150 con 0x7f6880102790 2024-09-19T09:30:50.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.144+0000 7f68767fc640 1 -- 172.21.15.28:0/162726351 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f687009b300 con 0x7f6880102790 2024-09-19T09:30:50.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.146+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 msgr2=0x7f686c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:50.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.146+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 0x7f686c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f687c005fd0 tx=0x7f687c005850 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.146+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 msgr2=0x7f6880074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.146+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f687000b4a0 tx=0x7f687000b960 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 shutdown_connections 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f686c0a3240 0x7f686c0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 --2- 172.21.15.28:0/162726351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 >> 172.21.15.28:0/162726351 conn(0x7f68800fd8c0 msgr2=0x7f68800fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:50.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 shutdown_connections 2024-09-19T09:30:50.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:50.147+0000 7f6887a33640 1 -- 172.21.15.28:0/162726351 wait complete. 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T09:30:50.755 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:50 smithi028 ceph-mon[30972]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T09:30:50.787 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T09:30:50.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T09:30:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:50 smithi078 ceph-mon[35619]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/162726351' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T09:30:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:50 smithi038 ceph-mon[35271]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T09:30:51.044 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:51.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 -- 172.21.15.28:0/3379481252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98101420 msgr2=0x7f0b9810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:51.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 --2- 172.21.15.28:0/3379481252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98101420 0x7f0b9810d950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009cd0 tx=0x7f0b9002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 -- 172.21.15.28:0/3379481252 shutdown_connections 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 --2- 172.21.15.28:0/3379481252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98100a80 0x7f0b98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 --2- 172.21.15.28:0/3379481252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98101420 0x7f0b9810d950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 --2- 172.21.15.28:0/3379481252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98106a80 0x7f0b98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 -- 172.21.15.28:0/3379481252 >> 172.21.15.28:0/3379481252 conn(0x7f0b980fc7a0 msgr2=0x7f0b980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 -- 172.21.15.28:0/3379481252 shutdown_connections 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.492+0000 7f0b9f123640 1 -- 172.21.15.28:0/3379481252 wait complete. 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 Processor -- start 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 -- start start 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98100a80 0x7f0b981992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:51.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 0x7f0b98199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98110ea0 con 0x7f0b98106a80 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98110d20 con 0x7f0b98101420 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9f123640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98111020 con 0x7f0b98100a80 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9d6c0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9d6c0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49780/0 (socket says 172.21.15.28:49780) 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9d6c0640 1 -- 172.21.15.28:0/5837541 learned_addr learned my addr 172.21.15.28:0/5837541 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b9cebf640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98100a80 0x7f0b981992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.493+0000 7f0b8ffff640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 0x7f0b98199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:51.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 -- 172.21.15.28:0/5837541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98100a80 msgr2=0x7f0b981992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98100a80 0x7f0b981992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 -- 172.21.15.28:0/5837541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 msgr2=0x7f0b98199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 0x7f0b98199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 -- 172.21.15.28:0/5837541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b981a4890 con 0x7f0b98106a80 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b8ffff640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 0x7f0b98199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9d6c0640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005cc0 tx=0x7f0b90003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004040 con 0x7f0b98106a80 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b900046c0 con 0x7f0b98106a80 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b981a4a60 con 0x7f0b98106a80 2024-09-19T09:30:51.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90046480 con 0x7f0b98106a80 2024-09-19T09:30:51.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.494+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b981a4f40 con 0x7f0b98106a80 2024-09-19T09:30:51.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.495+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b98100ee0 con 0x7f0b98106a80 2024-09-19T09:30:51.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.497+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0b900305a0 con 0x7f0b98106a80 2024-09-19T09:30:51.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.497+0000 7f0b8dffb640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 0x7f0b800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:51.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.497+0000 7f0b9cebf640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 0x7f0b800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:51.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.497+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0b9002e030 con 0x7f0b98106a80 2024-09-19T09:30:51.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.498+0000 7f0b9cebf640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 0x7f0b800a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400bbe0 tx=0x7f0b9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:51.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.502+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b9004b060 con 0x7f0b98106a80 2024-09-19T09:30:51.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:51.715+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0b981a5170 con 0x7f0b98106a80 2024-09-19T09:30:52.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.161+0000 7f0b8dffb640 1 -- 172.21.15.28:0/5837541 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f0b900bde10 con 0x7f0b98106a80 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 msgr2=0x7f0b800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 0x7f0b800a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400bbe0 tx=0x7f0b9400b390 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 msgr2=0x7f0b981a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005cc0 tx=0x7f0b90003970 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 shutdown_connections 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0b800a32c0 0x7f0b800a5780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b98101420 0x7f0b98199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b98106a80 0x7f0b981a4350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 --2- 172.21.15.28:0/5837541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b98100a80 0x7f0b981992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 >> 172.21.15.28:0/5837541 conn(0x7f0b980fc7a0 msgr2=0x7f0b980fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:52.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 shutdown_connections 2024-09-19T09:30:52.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:52.164+0000 7f0b9f123640 1 -- 172.21.15.28:0/5837541 wait complete. 2024-09-19T09:30:52.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:active 2024-09-19T09:30:52.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:30:52.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T09:30:52.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T09:30:52.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T09:30:52.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:52 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:30:52.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:active 2024-09-19T09:30:52.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:30:52.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T09:30:52.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T09:30:52.429 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T09:30:52.430 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:52 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] up:active 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T09:30:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:52 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:30:52.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T09:30:52.983 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:active 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T09:30:53.383 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:53 smithi028 ceph-mon[30972]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T09:30:53.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:30:53.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:30:53.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:active 2024-09-19T09:30:53.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:30:53.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T09:30:53.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T09:30:53.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:53 smithi038 ceph-mon[35271]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T09:30:53.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 -- 172.21.15.28:0/3939786492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 msgr2=0x7f979810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 --2- 172.21.15.28:0/3939786492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f979810dfb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9788009900 tx=0x7f978802a410 comp rx=0 tx=0).stop 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 -- 172.21.15.28:0/3939786492 shutdown_connections 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 --2- 172.21.15.28:0/3939786492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 0x7f9798110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 --2- 172.21.15.28:0/3939786492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f979810dfb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 --2- 172.21.15.28:0/3939786492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 0x7f9798100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 -- 172.21.15.28:0/3939786492 >> 172.21.15.28:0/3939786492 conn(0x7f97980fc7d0 msgr2=0x7f97980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 -- 172.21.15.28:0/3939786492 shutdown_connections 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.468+0000 7f97a070f640 1 -- 172.21.15.28:0/3939786492 wait complete. 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 Processor -- start 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 -- start start 2024-09-19T09:30:53.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 0x7f97981a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 0x7f979819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f97981142c0 con 0x7f9798101380 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9798114140 con 0x7f979810e4f0 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f97a070f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9798114440 con 0x7f97981009d0 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f979dcaa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f979dcaa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43572/0 (socket says 172.21.15.28:43572) 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f979dcaa640 1 -- 172.21.15.28:0/860032267 learned_addr learned my addr 172.21.15.28:0/860032267 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.469+0000 7f979ecac640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 0x7f979819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979e4ab640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 0x7f97981a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 -- 172.21.15.28:0/860032267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 msgr2=0x7f97981a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 0x7f97981a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 -- 172.21.15.28:0/860032267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 msgr2=0x7f979819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 0x7f979819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 -- 172.21.15.28:0/860032267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979819ace0 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f979dcaa640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9788005ab0 tx=0x7f97880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978803f030 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9788003ce0 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979819af70 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788035070 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.470+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a72c0 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.471+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979806a7d0 con 0x7f9798101380 2024-09-19T09:30:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.473+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9788042030 con 0x7f9798101380 2024-09-19T09:30:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.474+0000 7f978f7fe640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 0x7f97840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.474+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f97880f0350 con 0x7f9798101380 2024-09-19T09:30:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.474+0000 7f979e4ab640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 0x7f97840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.475+0000 7f979e4ab640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 0x7f97840a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9794009890 tx=0x7f9794008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.477+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97880bd120 con 0x7f9798101380 2024-09-19T09:30:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: pgmap v282: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:30:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/5837541' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:30:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] up:active 2024-09-19T09:30:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:30:53.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T09:30:53.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T09:30:53.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:53 smithi078 ceph-mon[35619]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T09:30:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:53.692+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9798100db0 con 0x7f9798101380 2024-09-19T09:30:54.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.181+0000 7f978f7fe640 1 -- 172.21.15.28:0/860032267 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f9798100db0 con 0x7f9798101380 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 msgr2=0x7f97840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 0x7f97840a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9794009890 tx=0x7f9794008040 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 msgr2=0x7f97981a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9788005ab0 tx=0x7f97880042f0 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 shutdown_connections 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f97840a32c0 0x7f97840a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f979810e4f0 0x7f979819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9798101380 0x7f97981a0930 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 --2- 172.21.15.28:0/860032267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97981009d0 0x7f97981a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 >> 172.21.15.28:0/860032267 conn(0x7f97980fc7d0 msgr2=0x7f97980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 shutdown_connections 2024-09-19T09:30:54.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:54.184+0000 7f97a070f640 1 -- 172.21.15.28:0/860032267 wait complete. 2024-09-19T09:30:54.494 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:30:54.494 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:30:54.494 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:30:54.495 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: Cluster is now healthy 2024-09-19T09:30:54.495 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T09:30:54.495 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:30:54.495 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T09:30:54.495 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:54 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:30:54.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:30:54.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: Cluster is now healthy 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T09:30:54.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:54 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: Cluster is now healthy 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T09:30:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:54 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:30:54.797 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T09:30:54.798 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T09:30:55.062 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:55.448 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-mon[30972]: pgmap v283: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-09-19T09:30:55.448 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:30:55.448 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:30:55.448 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-mon[30972]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:30:55.449 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 -- 172.21.15.28:0/2151881601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 msgr2=0x7f41701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 --2- 172.21.15.28:0/2151881601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f41701021c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4164009900 tx=0x7f416402a410 comp rx=0 tx=0).stop 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 -- 172.21.15.28:0/2151881601 shutdown_connections 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 --2- 172.21.15.28:0/2151881601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 --2- 172.21.15.28:0/2151881601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 0x7f41701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 --2- 172.21.15.28:0/2151881601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f41701021c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 -- 172.21.15.28:0/2151881601 >> 172.21.15.28:0/2151881601 conn(0x7f41700fd8c0 msgr2=0x7f41700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 -- 172.21.15.28:0/2151881601 shutdown_connections 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.532+0000 7f4176e98640 1 -- 172.21.15.28:0/2151881601 wait complete. 2024-09-19T09:30:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 Processor -- start 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 -- start start 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 0x7f4170198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4170111f70 con 0x7f4170101de0 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4170111df0 con 0x7f4170106f10 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4176e98640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41701120f0 con 0x7f4170102790 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43592/0 (socket says 172.21.15.28:43592) 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 -- 172.21.15.28:0/1869802048 learned_addr learned my addr 172.21.15.28:0/1869802048 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f416ffff640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 0x7f4170198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4175435640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 -- 172.21.15.28:0/1869802048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 msgr2=0x7f4170198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 0x7f4170198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 -- 172.21.15.28:0/1869802048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 msgr2=0x7f417019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4174c34640 1 -- 172.21.15.28:0/1869802048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417019d3e0 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.533+0000 7f4175435640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f4174c34640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4164006010 tx=0x7f41640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416403f030 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417019d670 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4164003ce0 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164035070 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.534+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41701a4fb0 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.535+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41701032c0 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.536+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4164042030 con 0x7f4170101de0 2024-09-19T09:30:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.537+0000 7f416dffb640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 0x7f41600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.537+0000 7f416ffff640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 0x7f41600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.537+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f41640efd70 con 0x7f4170101de0 2024-09-19T09:30:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.537+0000 7f416ffff640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 0x7f41600a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41700fdd50 tx=0x7f415c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.541+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41640bcc00 con 0x7f4170101de0 2024-09-19T09:30:55.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:55 smithi078 ceph-mon[35619]: pgmap v283: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-09-19T09:30:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:55 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:30:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:55 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:30:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:55 smithi078 ceph-mon[35619]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:30:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:55 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:30:55.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:55 smithi038 ceph-mon[35271]: pgmap v283: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-09-19T09:30:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:55 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/860032267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:30:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:55 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:30:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:55 smithi038 ceph-mon[35271]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:30:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:55 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:30:55.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.739+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4170062630 con 0x7f41600a3240 2024-09-19T09:30:55.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.770+0000 7f416dffb640 1 -- 172.21.15.28:0/1869802048 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4170062630 con 0x7f41600a3240 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 msgr2=0x7f41600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 0x7f41600a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41700fdd50 tx=0x7f415c00a250 comp rx=0 tx=0).stop 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 msgr2=0x7f4170198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4164006010 tx=0x7f41640042f0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 shutdown_connections 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41600a3240 0x7f41600a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4170106f10 0x7f417019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4170102790 0x7f4170198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 --2- 172.21.15.28:0/1869802048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4170101de0 0x7f4170198430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:55.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 >> 172.21.15.28:0/1869802048 conn(0x7f41700fd8c0 msgr2=0x7f41700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:55.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 shutdown_connections 2024-09-19T09:30:55.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:55.772+0000 7f4176e98640 1 -- 172.21.15.28:0/1869802048 wait complete. 2024-09-19T09:30:56.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:30:55.744+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:30:56.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:30:55.744+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:30:56.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:30:55.744+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:30:56.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:30:55.744+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:30:56.014 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:30:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:30:55.744+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:30:56.394 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T09:30:56.656 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- 172.21.15.28:0/2729165200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c100a80 msgr2=0x7fd15c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2729165200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c100a80 0x7fd15c100ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009930 tx=0x7fd14c02a420 comp rx=0 tx=0).stop 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- 172.21.15.28:0/2729165200 shutdown_connections 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2729165200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c101420 0x7fd15c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2729165200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c100a80 0x7fd15c100ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2729165200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c106a80 0x7fd15c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- 172.21.15.28:0/2729165200 >> 172.21.15.28:0/2729165200 conn(0x7fd15c0fc7a0 msgr2=0x7fd15c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- 172.21.15.28:0/2729165200 shutdown_connections 2024-09-19T09:30:57.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- 172.21.15.28:0/2729165200 wait complete. 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 Processor -- start 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.091+0000 7fd163ea2640 1 -- start start 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c100a80 0x7fd15c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c106a80 0x7fd15c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c110ed0 con 0x7fd15c101420 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c110d50 con 0x7fd15c100a80 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c111050 con 0x7fd15c106a80 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43624/0 (socket says 172.21.15.28:43624) 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 -- 172.21.15.28:0/2056889376 learned_addr learned my addr 172.21.15.28:0/2056889376 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:57.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd161c3e640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c100a80 0x7fd15c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16243f640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c106a80 0x7fd15c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 -- 172.21.15.28:0/2056889376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c106a80 msgr2=0x7fd15c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c106a80 0x7fd15c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 -- 172.21.15.28:0/2056889376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c100a80 msgr2=0x7fd15c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c100a80 0x7fd15c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 -- 172.21.15.28:0/2056889376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c19d440 con 0x7fd15c101420 2024-09-19T09:30:57.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd16143d640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a940 tx=0x7fd14c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:57.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c03f070 con 0x7fd15c101420 2024-09-19T09:30:57.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c031c90 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c19d6d0 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.092+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c035030 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.093+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1a5010 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.094+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c1021c0 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.095+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd14c042070 con 0x7fd15c101420 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.096+0000 7fd152ffd640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 0x7fd1480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.096+0000 7fd161c3e640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 0x7fd1480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:57.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.096+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd14c0efe60 con 0x7fd15c101420 2024-09-19T09:30:57.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.096+0000 7fd161c3e640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 0x7fd1480a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd15800bb60 tx=0x7fd15800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:57.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.100+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd14c0bc680 con 0x7fd15c101420 2024-09-19T09:30:57.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.303+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd15c062630 con 0x7fd1480a32c0 2024-09-19T09:30:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.329+0000 7fd152ffd640 1 -- 172.21.15.28:0/2056889376 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd15c062630 con 0x7fd1480a32c0 2024-09-19T09:30:57.331 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:57 smithi028 ceph-mon[30972]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-19T09:30:57.331 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:57 smithi028 ceph-mon[30972]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 msgr2=0x7fd1480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 0x7fd1480a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd15800bb60 tx=0x7fd15800b390 comp rx=0 tx=0).stop 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 msgr2=0x7fd15c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a940 tx=0x7fd14c003b90 comp rx=0 tx=0).stop 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 shutdown_connections 2024-09-19T09:30:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1480a32c0 0x7fd1480a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd15c106a80 0x7fd15c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd15c101420 0x7fd15c1989d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 --2- 172.21.15.28:0/2056889376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd15c100a80 0x7fd15c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 >> 172.21.15.28:0/2056889376 conn(0x7fd15c0fc7a0 msgr2=0x7fd15c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 shutdown_connections 2024-09-19T09:30:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:57.332+0000 7fd163ea2640 1 -- 172.21.15.28:0/2056889376 wait complete. 2024-09-19T09:30:57.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:57 smithi078 ceph-mon[35619]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-19T09:30:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:57 smithi078 ceph-mon[35619]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:57.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:57 smithi038 ceph-mon[35271]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-19T09:30:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:57 smithi038 ceph-mon[35271]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:57.958 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T09:30:58.223 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:58.457 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:58 smithi028 ceph-mon[30972]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:58.577 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:58 smithi038 ceph-mon[35271]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:58.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:58 smithi078 ceph-mon[35619]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:30:58.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 -- 172.21.15.28:0/3231063528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620103660 msgr2=0x7fc62010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:58.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3231063528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620103660 0x7fc62010bb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc614009900 tx=0x7fc61402a480 comp rx=0 tx=0).stop 2024-09-19T09:30:58.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 -- 172.21.15.28:0/3231063528 shutdown_connections 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3231063528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62010c170 0x7fc62010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3231063528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620103660 0x7fc62010bb50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3231063528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc620102d40 0x7fc620103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 -- 172.21.15.28:0/3231063528 >> 172.21.15.28:0/3231063528 conn(0x7fc6200fc7a0 msgr2=0x7fc6200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 -- 172.21.15.28:0/3231063528 shutdown_connections 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.672+0000 7fc6277a5640 1 -- 172.21.15.28:0/3231063528 wait complete. 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 Processor -- start 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 -- start start 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 0x7fc620198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:58.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc62010c170 0x7fc62019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc620111f70 con 0x7fc620102d40 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc620111df0 con 0x7fc62010c170 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc6277a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201120f0 con 0x7fc620103660 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc625541640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc625541640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43648/0 (socket says 172.21.15.28:43648) 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc625541640 1 -- 172.21.15.28:0/3403134781 learned_addr learned my addr 172.21.15.28:0/3403134781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:30:58.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.673+0000 7fc625d42640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc62010c170 0x7fc62019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc624d40640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 0x7fc620198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 -- 172.21.15.28:0/3403134781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 msgr2=0x7fc620198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 0x7fc620198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 -- 172.21.15.28:0/3403134781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc62010c170 msgr2=0x7fc62019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc62010c170 0x7fc62019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 -- 172.21.15.28:0/3403134781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019d3e0 con 0x7fc620102d40 2024-09-19T09:30:58.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc624d40640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 0x7fc620198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:58.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc625541640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00d950 tx=0x7fc61c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c017070 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c009c30 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019d6d0 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00f6b0 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.674+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a4fb0 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.675+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620103120 con 0x7fc620102d40 2024-09-19T09:30:58.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.676+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc61c0109d0 con 0x7fc620102d40 2024-09-19T09:30:58.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.677+0000 7fc6127fc640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 0x7fc60c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:58.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.677+0000 7fc624d40640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 0x7fc60c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:58.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.677+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc61c0cf580 con 0x7fc620102d40 2024-09-19T09:30:58.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.678+0000 7fc624d40640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 0x7fc60c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc62010d6b0 tx=0x7fc614005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:58.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.681+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc61c009de0 con 0x7fc620102d40 2024-09-19T09:30:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.885+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc620062630 con 0x7fc60c0a32d0 2024-09-19T09:30:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.890+0000 7fc6127fc640 1 -- 172.21.15.28:0/3403134781 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc620062630 con 0x7fc60c0a32d0 2024-09-19T09:30:58.892 INFO:teuthology.orchestra.run.smithi028.stdout:/volumes/qa/sv_0/64059f4e-529c-40a7-a1ce-0875a37482cd 2024-09-19T09:30:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 msgr2=0x7fc60c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 0x7fc60c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc62010d6b0 tx=0x7fc614005a20 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 msgr2=0x7fc620198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00d950 tx=0x7fc61c00de10 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 shutdown_connections 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc60c0a32d0 0x7fc60c0a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc62010c170 0x7fc62019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620103660 0x7fc620198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 --2- 172.21.15.28:0/3403134781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc620102d40 0x7fc620198430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 >> 172.21.15.28:0/3403134781 conn(0x7fc6200fc7a0 msgr2=0x7fc6200fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 shutdown_connections 2024-09-19T09:30:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:30:58.892+0000 7fc6277a5640 1 -- 172.21.15.28:0/3403134781 wait complete. 2024-09-19T09:30:59.475 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:59 smithi028 ceph-mon[30972]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 17 op/s 2024-09-19T09:30:59.475 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:30:59 smithi028 ceph-mon[30972]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:30:59.505 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T09:30:59.508 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:59 smithi078 ceph-mon[35619]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 17 op/s 2024-09-19T09:30:59.509 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:30:59 smithi078 ceph-mon[35619]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:30:59.588 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:59 smithi038 ceph-mon[35271]: pgmap v285: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 17 op/s 2024-09-19T09:30:59.589 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:30:59 smithi038 ceph-mon[35271]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:30:59.763 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:00.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 -- 172.21.15.28:0/1932920148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5008074620 msgr2=0x7f5008074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:00.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 --2- 172.21.15.28:0/1932920148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5008074620 0x7f5008074a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5004009900 tx=0x7f500402a480 comp rx=0 tx=0).stop 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 -- 172.21.15.28:0/1932920148 shutdown_connections 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 --2- 172.21.15.28:0/1932920148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f500810e3c0 0x7f5008110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 --2- 172.21.15.28:0/1932920148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 0x7f500810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 --2- 172.21.15.28:0/1932920148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5008074620 0x7f5008074a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 -- 172.21.15.28:0/1932920148 >> 172.21.15.28:0/1932920148 conn(0x7f50080fd8c0 msgr2=0x7f50080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 -- 172.21.15.28:0/1932920148 shutdown_connections 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.242+0000 7f500f2fc640 1 -- 172.21.15.28:0/1932920148 wait complete. 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 Processor -- start 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 -- start start 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 0x7f500819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 0x7f500819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:00.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5008114170 con 0x7f500810e3c0 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5008113ff0 con 0x7f5008074620 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500f2fc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50081142f0 con 0x7f5008074f40 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500d899640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500d098640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 0x7f500819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500d899640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43666/0 (socket says 172.21.15.28:43666) 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500d899640 1 -- 172.21.15.28:0/3641274713 learned_addr learned my addr 172.21.15.28:0/3641274713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.243+0000 7f500c897640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 0x7f500819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 -- 172.21.15.28:0/3641274713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 msgr2=0x7f500819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 0x7f500819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 -- 172.21.15.28:0/3641274713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 msgr2=0x7f500819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 0x7f500819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 -- 172.21.15.28:0/3641274713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500819f620 con 0x7f500810e3c0 2024-09-19T09:31:00.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d098640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 0x7f500819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:00.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500d899640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f500000eb10 tx=0x7f500000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:00.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000004b20 con 0x7f500810e3c0 2024-09-19T09:31:00.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000004cc0 con 0x7f500810e3c0 2024-09-19T09:31:00.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500819f910 con 0x7f500810e3c0 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000010760 con 0x7f500810e3c0 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.244+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50081a71f0 con 0x7f500810e3c0 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.245+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5008074a00 con 0x7f500810e3c0 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.246+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5000005020 con 0x7f500810e3c0 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.247+0000 7f4ffe7fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 0x7f4ff40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:00.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.247+0000 7f500d098640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 0x7f4ff40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:00.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.247+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f50000cfee0 con 0x7f500810e3c0 2024-09-19T09:31:00.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.248+0000 7f500d098640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 0x7f4ff40a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f50040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:00.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.251+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f500009cb50 con 0x7f500810e3c0 2024-09-19T09:31:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.452+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5008062630 con 0x7f4ff40a32d0 2024-09-19T09:31:00.454 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:00 smithi028 ceph-mon[30972]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:00.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.479+0000 7f4ffe7fc640 1 -- 172.21.15.28:0/3641274713 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5008062630 con 0x7f4ff40a32d0 2024-09-19T09:31:00.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 msgr2=0x7f4ff40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:00.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 0x7f4ff40a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f50040058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 msgr2=0x7f500819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:00.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f500000eb10 tx=0x7f500000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 shutdown_connections 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ff40a32d0 0x7f4ff40a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500810e3c0 0x7f500819ef40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5008074f40 0x7f500819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 --2- 172.21.15.28:0/3641274713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5008074620 0x7f500819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 >> 172.21.15.28:0/3641274713 conn(0x7f50080fd8c0 msgr2=0x7f50080fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 shutdown_connections 2024-09-19T09:31:00.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:00.481+0000 7f500f2fc640 1 -- 172.21.15.28:0/3641274713 wait complete. 2024-09-19T09:31:00.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:00 smithi078 ceph-mon[35619]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:00.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:00 smithi038 ceph-mon[35271]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:01.071 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T09:31:01.329 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:01.433 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:01 smithi028 ceph-mon[30972]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-19T09:31:01.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:01 smithi078 ceph-mon[35619]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-19T09:31:01.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:01 smithi038 ceph-mon[35271]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/3390947974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 msgr2=0x7f8dd010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/3390947974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd010ec40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009cd0 tx=0x7f8dc802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/3390947974 shutdown_connections 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/3390947974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd01036e0 0x7f8dd0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/3390947974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd010ec40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/3390947974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0102d30 0x7f8dd0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/3390947974 >> 172.21.15.28:0/3390947974 conn(0x7f8dd00fc7d0 msgr2=0x7f8dd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/3390947974 shutdown_connections 2024-09-19T09:31:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.754+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/3390947974 wait complete. 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 Processor -- start 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 -- start start 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd0102d30 0x7f8dd0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd01036e0 0x7f8dd0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01120a0 con 0x7f8dd01082a0 2024-09-19T09:31:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0111f20 con 0x7f8dd0102d30 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd6e3c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0112220 con 0x7f8dd01036e0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43690/0 (socket says 172.21.15.28:43690) 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 -- 172.21.15.28:0/4143901517 learned_addr learned my addr 172.21.15.28:0/4143901517 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dcffff640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd01036e0 0x7f8dd0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd4bd8640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd0102d30 0x7f8dd0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 -- 172.21.15.28:0/4143901517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd01036e0 msgr2=0x7f8dd0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd01036e0 0x7f8dd0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 -- 172.21.15.28:0/4143901517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd0102d30 msgr2=0x7f8dd0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd0102d30 0x7f8dd0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.755+0000 7f8dd53d9640 1 -- 172.21.15.28:0/4143901517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd01a6a30 con 0x7f8dd01082a0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dd53d9640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8005cc0 tx=0x7f8dc8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8004040 con 0x7f8dd01082a0 2024-09-19T09:31:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc80046c0 con 0x7f8dd01082a0 2024-09-19T09:31:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd01a6cc0 con 0x7f8dd01082a0 2024-09-19T09:31:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8046480 con 0x7f8dd01082a0 2024-09-19T09:31:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.756+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd01a7200 con 0x7f8dd01082a0 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.757+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd0103110 con 0x7f8dd01082a0 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.758+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8dc80305a0 con 0x7f8dd01082a0 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.759+0000 7f8dcdffb640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 0x7f8db80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.759+0000 7f8dd4bd8640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 0x7f8db80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.759+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f8dc802e030 con 0x7f8dd01082a0 2024-09-19T09:31:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.759+0000 7f8dd4bd8640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 0x7f8db80a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0005fd0 tx=0x7f8dc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:01.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.763+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8dc804b060 con 0x7f8dd01082a0 2024-09-19T09:31:01.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.967+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8dd0062630 con 0x7f8db80a32d0 2024-09-19T09:31:01.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.972+0000 7f8dcdffb640 1 -- 172.21.15.28:0/4143901517 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8dd0062630 con 0x7f8db80a32d0 2024-09-19T09:31:01.974 INFO:teuthology.orchestra.run.smithi028.stdout:/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391 2024-09-19T09:31:01.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 msgr2=0x7f8db80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:01.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 0x7f8db80a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0005fd0 tx=0x7f8dc000b390 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 msgr2=0x7f8dd0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8005cc0 tx=0x7f8dc8003970 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 shutdown_connections 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8db80a32d0 0x7f8db80a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8dd01082a0 0x7f8dd0101410 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd01036e0 0x7f8dd0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 --2- 172.21.15.28:0/4143901517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dd0102d30 0x7f8dd0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 >> 172.21.15.28:0/4143901517 conn(0x7f8dd00fc7d0 msgr2=0x7f8dd0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 shutdown_connections 2024-09-19T09:31:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:01.975+0000 7f8dd6e3c640 1 -- 172.21.15.28:0/4143901517 wait complete. 2024-09-19T09:31:02.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:02 smithi028 ceph-mon[30972]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:02.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:02 smithi078 ceph-mon[35619]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:02.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:31:02.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:02 smithi038 ceph-mon[35271]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:02.908 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:03.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:03 smithi028 ceph-mon[30972]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-19T09:31:03.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:03 smithi028 ceph-mon[30972]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 -- 172.21.15.28:0/3042559920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0101de0 msgr2=0x7fe2c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 --2- 172.21.15.28:0/3042559920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0101de0 0x7fe2c01021c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009900 tx=0x7fe2b002a410 comp rx=0 tx=0).stop 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 -- 172.21.15.28:0/3042559920 shutdown_connections 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 --2- 172.21.15.28:0/3042559920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0102790 0x7fe2c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 --2- 172.21.15.28:0/3042559920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 0x7fe2c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 --2- 172.21.15.28:0/3042559920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0101de0 0x7fe2c01021c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 -- 172.21.15.28:0/3042559920 >> 172.21.15.28:0/3042559920 conn(0x7fe2c00fd8c0 msgr2=0x7fe2c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:03.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 -- 172.21.15.28:0/3042559920 shutdown_connections 2024-09-19T09:31:03.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.315+0000 7fe2c7979640 1 -- 172.21.15.28:0/3042559920 wait complete. 2024-09-19T09:31:03.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 Processor -- start 2024-09-19T09:31:03.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 -- start start 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0101de0 0x7fe2c01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 0x7fe2c019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0112010 con 0x7fe2c0102790 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0111e90 con 0x7fe2c0101de0 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c7979640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0112190 con 0x7fe2c0106f10 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56526/0 (socket says 172.21.15.28:56526) 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 -- 172.21.15.28:0/2920700750 learned_addr learned my addr 172.21.15.28:0/2920700750 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c5715640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0101de0 0x7fe2c01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c5f16640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 0x7fe2c019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:03.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 -- 172.21.15.28:0/2920700750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 msgr2=0x7fe2c019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 0x7fe2c019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 -- 172.21.15.28:0/2920700750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0101de0 msgr2=0x7fe2c01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0101de0 0x7fe2c01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.316+0000 7fe2c4f14640 1 -- 172.21.15.28:0/2920700750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c019ad10 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2c4f14640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400b4a0 tx=0x7fe2b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4014070 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c019b000 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4010af0 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4017d10 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.317+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c01a72f0 con 0x7fe2c0102790 2024-09-19T09:31:03.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.318+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c0102240 con 0x7fe2c0102790 2024-09-19T09:31:03.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.319+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe2b4017070 con 0x7fe2c0102790 2024-09-19T09:31:03.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.320+0000 7fe2be7fc640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 0x7fe2a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:03.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.320+0000 7fe2c5715640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 0x7fe2a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:03.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.320+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe2b40ceb00 con 0x7fe2c0102790 2024-09-19T09:31:03.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.320+0000 7fe2c5715640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 0x7fe2a80a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:03.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.324+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2b409b330 con 0x7fe2c0102790 2024-09-19T09:31:03.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.537+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c01a7580 con 0x7fe2c0102790 2024-09-19T09:31:03.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.538+0000 7fe2be7fc640 1 -- 172.21.15.28:0/2920700750 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7fe2b409f630 con 0x7fe2c0102790 2024-09-19T09:31:03.540 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:03.540 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:31:03.540 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 msgr2=0x7fe2a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 0x7fe2a80a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b00058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 msgr2=0x7fe2c01a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400b4a0 tx=0x7fe2b400b960 comp rx=0 tx=0).stop 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 shutdown_connections 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe2a80a32d0 0x7fe2a80a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2c0106f10 0x7fe2c019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0102790 0x7fe2c01a0960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 --2- 172.21.15.28:0/2920700750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2c0101de0 0x7fe2c01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 >> 172.21.15.28:0/2920700750 conn(0x7fe2c00fd8c0 msgr2=0x7fe2c00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 shutdown_connections 2024-09-19T09:31:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:03.540+0000 7fe2c7979640 1 -- 172.21.15.28:0/2920700750 wait complete. 2024-09-19T09:31:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:03 smithi078 ceph-mon[35619]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-19T09:31:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:03 smithi078 ceph-mon[35619]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:03.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:03 smithi038 ceph-mon[35271]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-19T09:31:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:03 smithi038 ceph-mon[35271]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:31:04.161 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:31:04.420 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:04.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:04 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2920700750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:04.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:04 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2920700750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:04.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:04 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2920700750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:04.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 -- 172.21.15.28:0/2067320651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 msgr2=0x7f9730100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:04.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 --2- 172.21.15.28:0/2067320651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f9730100ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9720009930 tx=0x7f972002a420 comp rx=0 tx=0).stop 2024-09-19T09:31:04.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 -- 172.21.15.28:0/2067320651 shutdown_connections 2024-09-19T09:31:04.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 --2- 172.21.15.28:0/2067320651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730101420 0x7f973010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:04.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 --2- 172.21.15.28:0/2067320651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f9730100ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 --2- 172.21.15.28:0/2067320651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730106a80 0x7f9730106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 -- 172.21.15.28:0/2067320651 >> 172.21.15.28:0/2067320651 conn(0x7f97300fc7a0 msgr2=0x7f97300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 -- 172.21.15.28:0/2067320651 shutdown_connections 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.848+0000 7f973755b640 1 -- 172.21.15.28:0/2067320651 wait complete. 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 Processor -- start 2024-09-19T09:31:04.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 -- start start 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730101420 0x7f97300fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730106a80 0x7f97300fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9730110f20 con 0x7f9730100a80 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9730110da0 con 0x7f9730101420 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f973755b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97301110a0 con 0x7f9730106a80 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56540/0 (socket says 172.21.15.28:56540) 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 -- 172.21.15.28:0/671056170 learned_addr learned my addr 172.21.15.28:0/671056170 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f9735af8640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730106a80 0x7f97300fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f9734af6640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730101420 0x7f97300fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 -- 172.21.15.28:0/671056170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730106a80 msgr2=0x7f97300fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730106a80 0x7f97300fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 -- 172.21.15.28:0/671056170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730101420 msgr2=0x7f97300fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730101420 0x7f97300fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.849+0000 7f97352f7640 1 -- 172.21.15.28:0/671056170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301a6a90 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f97352f7640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f972c00b4a0 tx=0x7f972c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c014070 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972c010af0 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c017d10 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97301a6d80 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.850+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97301a7230 con 0x7f9730100a80 2024-09-19T09:31:04.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.851+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9730100f60 con 0x7f9730100a80 2024-09-19T09:31:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.852+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f972c017070 con 0x7f9730100a80 2024-09-19T09:31:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.853+0000 7f97267fc640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 0x7f971c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.853+0000 7f9734af6640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 0x7f971c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.853+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f972c0ceab0 con 0x7f9730100a80 2024-09-19T09:31:04.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.853+0000 7f9734af6640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 0x7f971c0a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:04.857+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f972c09b930 con 0x7f9730100a80 2024-09-19T09:31:05.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.034+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97300fe340 con 0x7f9730100a80 2024-09-19T09:31:05.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.036+0000 7f97267fc640 1 -- 172.21.15.28:0/671056170 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22195 (secure 0 0 0) 0x7f972c09b230 con 0x7f9730100a80 2024-09-19T09:31:05.038 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:05.039 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":80,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:30:43.054505+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:31:05.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 msgr2=0x7f971c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 0x7f971c0a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9720000c00 tx=0x7f9720031490 comp rx=0 tx=0).stop 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 msgr2=0x7f97300fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f972c00b4a0 tx=0x7f972c00b960 comp rx=0 tx=0).stop 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 shutdown_connections 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f971c0a3350 0x7f971c0a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:05.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9730101420 0x7f97300fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:05.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9730106a80 0x7f97300fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:05.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 --2- 172.21.15.28:0/671056170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9730100a80 0x7f97300fce10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:05.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 >> 172.21.15.28:0/671056170 conn(0x7f97300fc7a0 msgr2=0x7f973006aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:05.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 shutdown_connections 2024-09-19T09:31:05.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:05.038+0000 7f973755b640 1 -- 172.21.15.28:0/671056170 wait complete. 2024-09-19T09:31:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:05 smithi028 ceph-mon[30972]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-19T09:31:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:05 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/671056170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:05 smithi078 ceph-mon[35619]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-19T09:31:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:05 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/671056170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:05.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:05 smithi038 ceph-mon[35271]: pgmap v288: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-19T09:31:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:05 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/671056170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:05.807 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:31:05.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:31:06.070 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 -- 172.21.15.28:0/3696306515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 msgr2=0x7f89f0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/3696306515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0103120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009900 tx=0x7f89e002a480 comp rx=0 tx=0).stop 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 -- 172.21.15.28:0/3696306515 shutdown_connections 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/3696306515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f010c170 0x7f89f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/3696306515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f0103660 0x7f89f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/3696306515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0103120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 -- 172.21.15.28:0/3696306515 >> 172.21.15.28:0/3696306515 conn(0x7f89f00fc7a0 msgr2=0x7f89f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 -- 172.21.15.28:0/3696306515 shutdown_connections 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.487+0000 7f89f5cf1640 1 -- 172.21.15.28:0/3696306515 wait complete. 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 Processor -- start 2024-09-19T09:31:06.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 -- start start 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f0103660 0x7f89f0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f010c170 0x7f89f019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0111fc0 con 0x7f89f0102d40 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0111e40 con 0x7f89f0103660 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.488+0000 7f89f5cf1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0112140 con 0x7f89f010c170 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:06.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56566/0 (socket says 172.21.15.28:56566) 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 -- 172.21.15.28:0/1951929846 learned_addr learned my addr 172.21.15.28:0/1951929846 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89eeffd640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f0103660 0x7f89f0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89effff640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f010c170 0x7f89f019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 -- 172.21.15.28:0/1951929846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f010c170 msgr2=0x7f89f019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f010c170 0x7f89f019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 -- 172.21.15.28:0/1951929846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f0103660 msgr2=0x7f89f0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f0103660 0x7f89f0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 -- 172.21.15.28:0/1951929846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019d4b0 con 0x7f89f0102d40 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ef7fe640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89e0004770 tx=0x7f89e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e003f070 con 0x7f89f0102d40 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e0031ca0 con 0x7f89f0102d40 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019d740 con 0x7f89f0102d40 2024-09-19T09:31:06.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.489+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0035030 con 0x7f89f0102d40 2024-09-19T09:31:06.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.490+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f01a5080 con 0x7f89f0102d40 2024-09-19T09:31:06.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.491+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f0103120 con 0x7f89f0102d40 2024-09-19T09:31:06.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.492+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f89e0042070 con 0x7f89f0102d40 2024-09-19T09:31:06.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.493+0000 7f89ecff9640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 0x7f89d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:06.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.493+0000 7f89eeffd640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 0x7f89d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:06.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.493+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f89e00eff90 con 0x7f89f0102d40 2024-09-19T09:31:06.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.493+0000 7f89eeffd640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 0x7f89d80a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89f010d5d0 tx=0x7f89e400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:06.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.497+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89e00bc6b0 con 0x7f89f0102d40 2024-09-19T09:31:06.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.672+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89f0062630 con 0x7f89d80a3350 2024-09-19T09:31:06.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.679+0000 7f89ecff9640 1 -- 172.21.15.28:0/1951929846 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258755 (secure 0 0 0) 0x7f89f0062630 con 0x7f89d80a3350 2024-09-19T09:31:06.684 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:06.685 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:31:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 msgr2=0x7f89d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 0x7f89d80a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89f010d5d0 tx=0x7f89e400b2e0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 msgr2=0x7f89f0198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89e0004770 tx=0x7f89e0003c10 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 shutdown_connections 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f89d80a3350 0x7f89d80a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f0103660 0x7f89f0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89f010c170 0x7f89f019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 --2- 172.21.15.28:0/1951929846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f0102d40 0x7f89f0198500 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 >> 172.21.15.28:0/1951929846 conn(0x7f89f00fc7a0 msgr2=0x7f89f00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.682+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 shutdown_connections 2024-09-19T09:31:06.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:06.683+0000 7f89f5cf1640 1 -- 172.21.15.28:0/1951929846 wait complete. 2024-09-19T09:31:07.342 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-19T09:31:05.400447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639268,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":258,"num_read_kb":233,"num_write":410,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":337,"ondisk_log_size":337,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365692,"kb_used_data":12284,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576132,"statfs":{"total":1151940427776,"available":1151565959168,"internally_reserved":0,"allocated":12578816,"data_stored":6885483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44260,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":112,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565004+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:30:43.565004+0000","last_peered":"2024-09-19T09:30:43.565004+0000","last_clean":"2024-09-19T09:30:43.565004+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:30:43.565004+0000","last_undegraded":"2024-09-19T09:30:43.565004+0000","last_fullsized":"2024-09-19T09:30:43.565004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563865+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:30:43.563865+0000","last_peered":"2024-09-19T09:30:43.563865+0000","last_clean":"2024-09-19T09:30:43.563865+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:30:43.563865+0000","last_undegraded":"2024-09-19T09:30:43.563865+0000","last_fullsized":"2024-09-19T09:30:43.563865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564249+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:30:43.564249+0000","last_peered":"2024-09-19T09:30:43.564249+0000","last_clean":"2024-09-19T09:30:43.564249+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:30:43.564249+0000","last_undegraded":"2024-09-19T09:30:43.564249+0000","last_fullsized":"2024-09-19T09:30:43.564249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561198+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:30:43.561198+0000","last_peered":"2024-09-19T09:30:43.561198+0000","last_clean":"2024-09-19T09:30:43.561198+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:30:43.561198+0000","last_undegraded":"2024-09-19T09:30:43.561198+0000","last_fullsized":"2024-09-19T09:30:43.561198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561252+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:30:43.561252+0000","last_peered":"2024-09-19T09:30:43.561252+0000","last_clean":"2024-09-19T09:30:43.561252+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:30:43.561252+0000","last_undegraded":"2024-09-19T09:30:43.561252+0000","last_fullsized":"2024-09-19T09:30:43.561252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.547978+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:30:43.547978+0000","last_peered":"2024-09-19T09:30:43.547978+0000","last_clean":"2024-09-19T09:30:43.547978+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:30:43.547978+0000","last_undegraded":"2024-09-19T09:30:43.547978+0000","last_fullsized":"2024-09-19T09:30:43.547978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562172+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:30:43.562172+0000","last_peered":"2024-09-19T09:30:43.562172+0000","last_clean":"2024-09-19T09:30:43.562172+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:30:43.562172+0000","last_undegraded":"2024-09-19T09:30:43.562172+0000","last_fullsized":"2024-09-19T09:30:43.562172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542518+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:30:43.542518+0000","last_peered":"2024-09-19T09:30:43.542518+0000","last_clean":"2024-09-19T09:30:43.542518+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:30:43.542518+0000","last_undegraded":"2024-09-19T09:30:43.542518+0000","last_fullsized":"2024-09-19T09:30:43.542518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562779+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:30:43.562779+0000","last_peered":"2024-09-19T09:30:43.562779+0000","last_clean":"2024-09-19T09:30:43.562779+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:30:43.562779+0000","last_undegraded":"2024-09-19T09:30:43.562779+0000","last_fullsized":"2024-09-19T09:30:43.562779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548341+0000","last_change":"2024-09-19T09:30:42.564174+0000","last_active":"2024-09-19T09:30:43.548341+0000","last_peered":"2024-09-19T09:30:43.548341+0000","last_clean":"2024-09-19T09:30:43.548341+0000","last_became_active":"2024-09-19T09:30:42.563790+0000","last_became_peered":"2024-09-19T09:30:42.563790+0000","last_unstale":"2024-09-19T09:30:43.548341+0000","last_undegraded":"2024-09-19T09:30:43.548341+0000","last_fullsized":"2024-09-19T09:30:43.548341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:22.463896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564526+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:30:43.564526+0000","last_peered":"2024-09-19T09:30:43.564526+0000","last_clean":"2024-09-19T09:30:43.564526+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:30:43.564526+0000","last_undegraded":"2024-09-19T09:30:43.564526+0000","last_fullsized":"2024-09-19T09:30:43.564526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561947+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:30:43.561947+0000","last_peered":"2024-09-19T09:30:43.561947+0000","last_clean":"2024-09-19T09:30:43.561947+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:30:43.561947+0000","last_undegraded":"2024-09-19T09:30:43.561947+0000","last_fullsized":"2024-09-19T09:30:43.561947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561657+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:30:43.561657+0000","last_peered":"2024-09-19T09:30:43.561657+0000","last_clean":"2024-09-19T09:30:43.561657+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:30:43.561657+0000","last_undegraded":"2024-09-19T09:30:43.561657+0000","last_fullsized":"2024-09-19T09:30:43.561657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562189+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:30:43.562189+0000","last_peered":"2024-09-19T09:30:43.562189+0000","last_clean":"2024-09-19T09:30:43.562189+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:30:43.562189+0000","last_undegraded":"2024-09-19T09:30:43.562189+0000","last_fullsized":"2024-09-19T09:30:43.562189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562374+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:30:43.562374+0000","last_peered":"2024-09-19T09:30:43.562374+0000","last_clean":"2024-09-19T09:30:43.562374+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:30:43.562374+0000","last_undegraded":"2024-09-19T09:30:43.562374+0000","last_fullsized":"2024-09-19T09:30:43.562374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560910+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:30:43.560910+0000","last_peered":"2024-09-19T09:30:43.560910+0000","last_clean":"2024-09-19T09:30:43.560910+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:30:43.560910+0000","last_undegraded":"2024-09-19T09:30:43.560910+0000","last_fullsized":"2024-09-19T09:30:43.560910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565188+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:30:43.565188+0000","last_peered":"2024-09-19T09:30:43.565188+0000","last_clean":"2024-09-19T09:30:43.565188+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:30:43.565188+0000","last_undegraded":"2024-09-19T09:30:43.565188+0000","last_fullsized":"2024-09-19T09:30:43.565188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560190+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:30:43.560190+0000","last_peered":"2024-09-19T09:30:43.560190+0000","last_clean":"2024-09-19T09:30:43.560190+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:30:43.560190+0000","last_undegraded":"2024-09-19T09:30:43.560190+0000","last_fullsized":"2024-09-19T09:30:43.560190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563025+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:30:43.563025+0000","last_peered":"2024-09-19T09:30:43.563025+0000","last_clean":"2024-09-19T09:30:43.563025+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:30:43.563025+0000","last_undegraded":"2024-09-19T09:30:43.563025+0000","last_fullsized":"2024-09-19T09:30:43.563025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564597+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:30:43.564597+0000","last_peered":"2024-09-19T09:30:43.564597+0000","last_clean":"2024-09-19T09:30:43.564597+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:30:43.564597+0000","last_undegraded":"2024-09-19T09:30:43.564597+0000","last_fullsized":"2024-09-19T09:30:43.564597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561822+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:30:43.561822+0000","last_peered":"2024-09-19T09:30:43.561822+0000","last_clean":"2024-09-19T09:30:43.561822+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:30:43.561822+0000","last_undegraded":"2024-09-19T09:30:43.561822+0000","last_fullsized":"2024-09-19T09:30:43.561822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561430+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:30:43.561430+0000","last_peered":"2024-09-19T09:30:43.561430+0000","last_clean":"2024-09-19T09:30:43.561430+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:30:43.561430+0000","last_undegraded":"2024-09-19T09:30:43.561430+0000","last_fullsized":"2024-09-19T09:30:43.561430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542541+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:30:43.542541+0000","last_peered":"2024-09-19T09:30:43.542541+0000","last_clean":"2024-09-19T09:30:43.542541+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:30:43.542541+0000","last_undegraded":"2024-09-19T09:30:43.542541+0000","last_fullsized":"2024-09-19T09:30:43.542541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544194+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:30:43.544194+0000","last_peered":"2024-09-19T09:30:43.544194+0000","last_clean":"2024-09-19T09:30:43.544194+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:30:43.544194+0000","last_undegraded":"2024-09-19T09:30:43.544194+0000","last_fullsized":"2024-09-19T09:30:43.544194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542969+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:30:43.542969+0000","last_peered":"2024-09-19T09:30:43.542969+0000","last_clean":"2024-09-19T09:30:43.542969+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:30:43.542969+0000","last_undegraded":"2024-09-19T09:30:43.542969+0000","last_fullsized":"2024-09-19T09:30:43.542969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565943+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:30:43.565943+0000","last_peered":"2024-09-19T09:30:43.565943+0000","last_clean":"2024-09-19T09:30:43.565943+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:30:43.565943+0000","last_undegraded":"2024-09-19T09:30:43.565943+0000","last_fullsized":"2024-09-19T09:30:43.565943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564893+0000","last_change":"2024-09-19T09:30:42.559055+0000","last_active":"2024-09-19T09:30:43.564893+0000","last_peered":"2024-09-19T09:30:43.564893+0000","last_clean":"2024-09-19T09:30:43.564893+0000","last_became_active":"2024-09-19T09:30:42.558537+0000","last_became_peered":"2024-09-19T09:30:42.558537+0000","last_unstale":"2024-09-19T09:30:43.564893+0000","last_undegraded":"2024-09-19T09:30:43.564893+0000","last_fullsized":"2024-09-19T09:30:43.564893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.777026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562079+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:30:43.562079+0000","last_peered":"2024-09-19T09:30:43.562079+0000","last_clean":"2024-09-19T09:30:43.562079+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:30:43.562079+0000","last_undegraded":"2024-09-19T09:30:43.562079+0000","last_fullsized":"2024-09-19T09:30:43.562079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564360+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:30:43.564360+0000","last_peered":"2024-09-19T09:30:43.564360+0000","last_clean":"2024-09-19T09:30:43.564360+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:30:43.564360+0000","last_undegraded":"2024-09-19T09:30:43.564360+0000","last_fullsized":"2024-09-19T09:30:43.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562024+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:30:43.562024+0000","last_peered":"2024-09-19T09:30:43.562024+0000","last_clean":"2024-09-19T09:30:43.562024+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:30:43.562024+0000","last_undegraded":"2024-09-19T09:30:43.562024+0000","last_fullsized":"2024-09-19T09:30:43.562024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563172+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:30:43.563172+0000","last_peered":"2024-09-19T09:30:43.563172+0000","last_clean":"2024-09-19T09:30:43.563172+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:30:43.563172+0000","last_undegraded":"2024-09-19T09:30:43.563172+0000","last_fullsized":"2024-09-19T09:30:43.563172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563982+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:30:43.563982+0000","last_peered":"2024-09-19T09:30:43.563982+0000","last_clean":"2024-09-19T09:30:43.563982+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:30:43.563982+0000","last_undegraded":"2024-09-19T09:30:43.563982+0000","last_fullsized":"2024-09-19T09:30:43.563982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560987+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:30:43.560987+0000","last_peered":"2024-09-19T09:30:43.560987+0000","last_clean":"2024-09-19T09:30:43.560987+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:30:43.560987+0000","last_undegraded":"2024-09-19T09:30:43.560987+0000","last_fullsized":"2024-09-19T09:30:43.560987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:59.206981+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:30:59.206981+0000","last_peered":"2024-09-19T09:30:59.206981+0000","last_clean":"2024-09-19T09:30:59.206981+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:30:59.206981+0000","last_undegraded":"2024-09-19T09:30:59.206981+0000","last_fullsized":"2024-09-19T09:30:59.206981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544051+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:30:43.544051+0000","last_peered":"2024-09-19T09:30:43.544051+0000","last_clean":"2024-09-19T09:30:43.544051+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:30:43.544051+0000","last_undegraded":"2024-09-19T09:30:43.544051+0000","last_fullsized":"2024-09-19T09:30:43.544051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184065+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:30:52.184065+0000","last_peered":"2024-09-19T09:30:52.184065+0000","last_clean":"2024-09-19T09:30:52.184065+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:30:52.184065+0000","last_undegraded":"2024-09-19T09:30:52.184065+0000","last_fullsized":"2024-09-19T09:30:52.184065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.163943+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:30:51.163943+0000","last_peered":"2024-09-19T09:30:51.163943+0000","last_clean":"2024-09-19T09:30:51.163943+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:30:51.163943+0000","last_undegraded":"2024-09-19T09:30:51.163943+0000","last_fullsized":"2024-09-19T09:30:51.163943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542591+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:30:43.542591+0000","last_peered":"2024-09-19T09:30:43.542591+0000","last_clean":"2024-09-19T09:30:43.542591+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:30:43.542591+0000","last_undegraded":"2024-09-19T09:30:43.542591+0000","last_fullsized":"2024-09-19T09:30:43.542591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187883+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:30:53.187883+0000","last_peered":"2024-09-19T09:30:53.187883+0000","last_clean":"2024-09-19T09:30:53.187883+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:30:53.187883+0000","last_undegraded":"2024-09-19T09:30:53.187883+0000","last_fullsized":"2024-09-19T09:30:53.187883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.559954+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:30:43.559954+0000","last_peered":"2024-09-19T09:30:43.559954+0000","last_clean":"2024-09-19T09:30:43.559954+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:30:43.559954+0000","last_undegraded":"2024-09-19T09:30:43.559954+0000","last_fullsized":"2024-09-19T09:30:43.559954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.208238+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:31:03.208238+0000","last_peered":"2024-09-19T09:31:03.208238+0000","last_clean":"2024-09-19T09:31:03.208238+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:31:03.208238+0000","last_undegraded":"2024-09-19T09:31:03.208238+0000","last_fullsized":"2024-09-19T09:31:03.208238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562997+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:30:43.562997+0000","last_peered":"2024-09-19T09:30:43.562997+0000","last_clean":"2024-09-19T09:30:43.562997+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:30:43.562997+0000","last_undegraded":"2024-09-19T09:30:43.562997+0000","last_fullsized":"2024-09-19T09:30:43.562997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187517+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:30:53.187517+0000","last_peered":"2024-09-19T09:30:53.187517+0000","last_clean":"2024-09-19T09:30:53.187517+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:30:53.187517+0000","last_undegraded":"2024-09-19T09:30:53.187517+0000","last_fullsized":"2024-09-19T09:30:53.187517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564521+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:30:43.564521+0000","last_peered":"2024-09-19T09:30:43.564521+0000","last_clean":"2024-09-19T09:30:43.564521+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:30:43.564521+0000","last_undegraded":"2024-09-19T09:30:43.564521+0000","last_fullsized":"2024-09-19T09:30:43.564521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.182130+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:30:52.182130+0000","last_peered":"2024-09-19T09:30:52.182130+0000","last_clean":"2024-09-19T09:30:52.182130+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:30:52.182130+0000","last_undegraded":"2024-09-19T09:30:52.182130+0000","last_fullsized":"2024-09-19T09:30:52.182130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562134+0000","last_change":"2024-09-19T09:30:42.563614+0000","last_active":"2024-09-19T09:30:43.562134+0000","last_peered":"2024-09-19T09:30:43.562134+0000","last_clean":"2024-09-19T09:30:43.562134+0000","last_became_active":"2024-09-19T09:30:42.563289+0000","last_became_peered":"2024-09-19T09:30:42.563289+0000","last_unstale":"2024-09-19T09:30:43.562134+0000","last_undegraded":"2024-09-19T09:30:43.562134+0000","last_fullsized":"2024-09-19T09:30:43.562134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:47.215006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'23","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.212519+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:31:03.212519+0000","last_peered":"2024-09-19T09:31:03.212519+0000","last_clean":"2024-09-19T09:31:03.212519+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:31:03.212519+0000","last_undegraded":"2024-09-19T09:31:03.212519+0000","last_fullsized":"2024-09-19T09:31:03.212519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":52,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561453+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:30:43.561453+0000","last_peered":"2024-09-19T09:30:43.561453+0000","last_clean":"2024-09-19T09:30:43.561453+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:30:43.561453+0000","last_undegraded":"2024-09-19T09:30:43.561453+0000","last_fullsized":"2024-09-19T09:30:43.561453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.191692+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:30:54.191692+0000","last_peered":"2024-09-19T09:30:54.191692+0000","last_clean":"2024-09-19T09:30:54.191692+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:30:54.191692+0000","last_undegraded":"2024-09-19T09:30:54.191692+0000","last_fullsized":"2024-09-19T09:30:54.191692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561229+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:30:43.561229+0000","last_peered":"2024-09-19T09:30:43.561229+0000","last_clean":"2024-09-19T09:30:43.561229+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:30:43.561229+0000","last_undegraded":"2024-09-19T09:30:43.561229+0000","last_fullsized":"2024-09-19T09:30:43.561229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188935+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:30:53.188935+0000","last_peered":"2024-09-19T09:30:53.188935+0000","last_clean":"2024-09-19T09:30:53.188935+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:30:53.188935+0000","last_undegraded":"2024-09-19T09:30:53.188935+0000","last_fullsized":"2024-09-19T09:30:53.188935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564099+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:30:43.564099+0000","last_peered":"2024-09-19T09:30:43.564099+0000","last_clean":"2024-09-19T09:30:43.564099+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:30:43.564099+0000","last_undegraded":"2024-09-19T09:30:43.564099+0000","last_fullsized":"2024-09-19T09:30:43.564099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:00.206845+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:31:00.206845+0000","last_peered":"2024-09-19T09:31:00.206845+0000","last_clean":"2024-09-19T09:31:00.206845+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:31:00.206845+0000","last_undegraded":"2024-09-19T09:31:00.206845+0000","last_fullsized":"2024-09-19T09:31:00.206845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562555+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:30:43.562555+0000","last_peered":"2024-09-19T09:30:43.562555+0000","last_clean":"2024-09-19T09:30:43.562555+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:30:43.562555+0000","last_undegraded":"2024-09-19T09:30:43.562555+0000","last_fullsized":"2024-09-19T09:30:43.562555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186229+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:30:53.186229+0000","last_peered":"2024-09-19T09:30:53.186229+0000","last_clean":"2024-09-19T09:30:53.186229+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:30:53.186229+0000","last_undegraded":"2024-09-19T09:30:53.186229+0000","last_fullsized":"2024-09-19T09:30:53.186229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544131+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:30:43.544131+0000","last_peered":"2024-09-19T09:30:43.544131+0000","last_clean":"2024-09-19T09:30:43.544131+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:30:43.544131+0000","last_undegraded":"2024-09-19T09:30:43.544131+0000","last_fullsized":"2024-09-19T09:30:43.544131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.160939+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:30:53.160939+0000","last_peered":"2024-09-19T09:30:53.160939+0000","last_clean":"2024-09-19T09:30:53.160939+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:30:53.160939+0000","last_undegraded":"2024-09-19T09:30:53.160939+0000","last_fullsized":"2024-09-19T09:30:53.160939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564588+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:30:43.564588+0000","last_peered":"2024-09-19T09:30:43.564588+0000","last_clean":"2024-09-19T09:30:43.564588+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:30:43.564588+0000","last_undegraded":"2024-09-19T09:30:43.564588+0000","last_fullsized":"2024-09-19T09:30:43.564588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186972+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:30:53.186972+0000","last_peered":"2024-09-19T09:30:53.186972+0000","last_clean":"2024-09-19T09:30:53.186972+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:30:53.186972+0000","last_undegraded":"2024-09-19T09:30:53.186972+0000","last_fullsized":"2024-09-19T09:30:53.186972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564675+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:30:43.564675+0000","last_peered":"2024-09-19T09:30:43.564675+0000","last_clean":"2024-09-19T09:30:43.564675+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:30:43.564675+0000","last_undegraded":"2024-09-19T09:30:43.564675+0000","last_fullsized":"2024-09-19T09:30:43.564675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186823+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:30:53.186823+0000","last_peered":"2024-09-19T09:30:53.186823+0000","last_clean":"2024-09-19T09:30:53.186823+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:30:53.186823+0000","last_undegraded":"2024-09-19T09:30:53.186823+0000","last_fullsized":"2024-09-19T09:30:53.186823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564388+0000","last_change":"2024-09-19T09:30:42.558972+0000","last_active":"2024-09-19T09:30:43.564388+0000","last_peered":"2024-09-19T09:30:43.564388+0000","last_clean":"2024-09-19T09:30:43.564388+0000","last_became_active":"2024-09-19T09:30:42.558388+0000","last_became_peered":"2024-09-19T09:30:42.558388+0000","last_unstale":"2024-09-19T09:30:43.564388+0000","last_undegraded":"2024-09-19T09:30:43.564388+0000","last_fullsized":"2024-09-19T09:30:43.564388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:20.537069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.208965+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:31:03.208965+0000","last_peered":"2024-09-19T09:31:03.208965+0000","last_clean":"2024-09-19T09:31:03.208965+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:31:03.208965+0000","last_undegraded":"2024-09-19T09:31:03.208965+0000","last_fullsized":"2024-09-19T09:31:03.208965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564607+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:30:43.564607+0000","last_peered":"2024-09-19T09:30:43.564607+0000","last_clean":"2024-09-19T09:30:43.564607+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:30:43.564607+0000","last_undegraded":"2024-09-19T09:30:43.564607+0000","last_fullsized":"2024-09-19T09:30:43.564607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562904+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:30:43.562904+0000","last_peered":"2024-09-19T09:30:43.562904+0000","last_clean":"2024-09-19T09:30:43.562904+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:30:43.562904+0000","last_undegraded":"2024-09-19T09:30:43.562904+0000","last_fullsized":"2024-09-19T09:30:43.562904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192354+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:30:54.192354+0000","last_peered":"2024-09-19T09:30:54.192354+0000","last_clean":"2024-09-19T09:30:54.192354+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:30:54.192354+0000","last_undegraded":"2024-09-19T09:30:54.192354+0000","last_fullsized":"2024-09-19T09:30:54.192354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542792+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:30:43.542792+0000","last_peered":"2024-09-19T09:30:43.542792+0000","last_clean":"2024-09-19T09:30:43.542792+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:30:43.542792+0000","last_undegraded":"2024-09-19T09:30:43.542792+0000","last_fullsized":"2024-09-19T09:30:43.542792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.165255+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:30:51.165255+0000","last_peered":"2024-09-19T09:30:51.165255+0000","last_clean":"2024-09-19T09:30:51.165255+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:30:51.165255+0000","last_undegraded":"2024-09-19T09:30:51.165255+0000","last_fullsized":"2024-09-19T09:30:51.165255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561146+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:30:43.561146+0000","last_peered":"2024-09-19T09:30:43.561146+0000","last_clean":"2024-09-19T09:30:43.561146+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:30:43.561146+0000","last_undegraded":"2024-09-19T09:30:43.561146+0000","last_fullsized":"2024-09-19T09:30:43.561146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:01.207447+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:31:01.207447+0000","last_peered":"2024-09-19T09:31:01.207447+0000","last_clean":"2024-09-19T09:31:01.207447+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:31:01.207447+0000","last_undegraded":"2024-09-19T09:31:01.207447+0000","last_fullsized":"2024-09-19T09:31:01.207447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543968+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:30:43.543968+0000","last_peered":"2024-09-19T09:30:43.543968+0000","last_clean":"2024-09-19T09:30:43.543968+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:30:43.543968+0000","last_undegraded":"2024-09-19T09:30:43.543968+0000","last_fullsized":"2024-09-19T09:30:43.543968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:00.204704+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:31:00.204704+0000","last_peered":"2024-09-19T09:31:00.204704+0000","last_clean":"2024-09-19T09:31:00.204704+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:31:00.204704+0000","last_undegraded":"2024-09-19T09:31:00.204704+0000","last_fullsized":"2024-09-19T09:31:00.204704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548405+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:30:43.548405+0000","last_peered":"2024-09-19T09:30:43.548405+0000","last_clean":"2024-09-19T09:30:43.548405+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:30:43.548405+0000","last_undegraded":"2024-09-19T09:30:43.548405+0000","last_fullsized":"2024-09-19T09:30:43.548405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560204+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:30:43.560204+0000","last_peered":"2024-09-19T09:30:43.560204+0000","last_clean":"2024-09-19T09:30:43.560204+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:30:43.560204+0000","last_undegraded":"2024-09-19T09:30:43.560204+0000","last_fullsized":"2024-09-19T09:30:43.560204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:44.011551+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:30:44.011551+0000","last_peered":"2024-09-19T09:30:44.011551+0000","last_clean":"2024-09-19T09:30:44.011551+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:30:44.011551+0000","last_undegraded":"2024-09-19T09:30:44.011551+0000","last_fullsized":"2024-09-19T09:30:44.011551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188733+0000","last_change":"2024-09-19T09:30:40.592827+0000","last_active":"2024-09-19T09:30:53.188733+0000","last_peered":"2024-09-19T09:30:53.188733+0000","last_clean":"2024-09-19T09:30:53.188733+0000","last_became_active":"2024-09-19T09:30:40.591986+0000","last_became_peered":"2024-09-19T09:30:40.591986+0000","last_unstale":"2024-09-19T09:30:53.188733+0000","last_undegraded":"2024-09-19T09:30:53.188733+0000","last_fullsized":"2024-09-19T09:30:53.188733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:11.426185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562237+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:30:43.562237+0000","last_peered":"2024-09-19T09:30:43.562237+0000","last_clean":"2024-09-19T09:30:43.562237+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:30:43.562237+0000","last_undegraded":"2024-09-19T09:30:43.562237+0000","last_fullsized":"2024-09-19T09:30:43.562237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:57.560578+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:30:57.560578+0000","last_peered":"2024-09-19T09:30:57.560578+0000","last_clean":"2024-09-19T09:30:57.560578+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:30:57.560578+0000","last_undegraded":"2024-09-19T09:30:57.560578+0000","last_fullsized":"2024-09-19T09:30:57.560578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543123+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:30:43.543123+0000","last_peered":"2024-09-19T09:30:43.543123+0000","last_clean":"2024-09-19T09:30:43.543123+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:30:43.543123+0000","last_undegraded":"2024-09-19T09:30:43.543123+0000","last_fullsized":"2024-09-19T09:30:43.543123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186347+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:30:53.186347+0000","last_peered":"2024-09-19T09:30:53.186347+0000","last_clean":"2024-09-19T09:30:53.186347+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:30:53.186347+0000","last_undegraded":"2024-09-19T09:30:53.186347+0000","last_fullsized":"2024-09-19T09:30:53.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564238+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:30:43.564238+0000","last_peered":"2024-09-19T09:30:43.564238+0000","last_clean":"2024-09-19T09:30:43.564238+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:30:43.564238+0000","last_undegraded":"2024-09-19T09:30:43.564238+0000","last_fullsized":"2024-09-19T09:30:43.564238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155515+0000","last_change":"2024-09-19T09:30:40.592013+0000","last_active":"2024-09-19T09:30:50.155515+0000","last_peered":"2024-09-19T09:30:50.155515+0000","last_clean":"2024-09-19T09:30:50.155515+0000","last_became_active":"2024-09-19T09:30:40.591742+0000","last_became_peered":"2024-09-19T09:30:40.591742+0000","last_unstale":"2024-09-19T09:30:50.155515+0000","last_undegraded":"2024-09-19T09:30:50.155515+0000","last_fullsized":"2024-09-19T09:30:50.155515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:24.014799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560917+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:30:43.560917+0000","last_peered":"2024-09-19T09:30:43.560917+0000","last_clean":"2024-09-19T09:30:43.560917+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:30:43.560917+0000","last_undegraded":"2024-09-19T09:30:43.560917+0000","last_fullsized":"2024-09-19T09:30:43.560917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186945+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:30:53.186945+0000","last_peered":"2024-09-19T09:30:53.186945+0000","last_clean":"2024-09-19T09:30:53.186945+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:30:53.186945+0000","last_undegraded":"2024-09-19T09:30:53.186945+0000","last_fullsized":"2024-09-19T09:30:53.186945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564140+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:30:43.564140+0000","last_peered":"2024-09-19T09:30:43.564140+0000","last_clean":"2024-09-19T09:30:43.564140+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:30:43.564140+0000","last_undegraded":"2024-09-19T09:30:43.564140+0000","last_fullsized":"2024-09-19T09:30:43.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184164+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:30:52.184164+0000","last_peered":"2024-09-19T09:30:52.184164+0000","last_clean":"2024-09-19T09:30:52.184164+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:30:52.184164+0000","last_undegraded":"2024-09-19T09:30:52.184164+0000","last_fullsized":"2024-09-19T09:30:52.184164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542815+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:30:43.542815+0000","last_peered":"2024-09-19T09:30:43.542815+0000","last_clean":"2024-09-19T09:30:43.542815+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:30:43.542815+0000","last_undegraded":"2024-09-19T09:30:43.542815+0000","last_fullsized":"2024-09-19T09:30:43.542815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155146+0000","last_change":"2024-09-19T09:30:40.535163+0000","last_active":"2024-09-19T09:30:50.155146+0000","last_peered":"2024-09-19T09:30:50.155146+0000","last_clean":"2024-09-19T09:30:50.155146+0000","last_became_active":"2024-09-19T09:30:40.534756+0000","last_became_peered":"2024-09-19T09:30:40.534756+0000","last_unstale":"2024-09-19T09:30:50.155146+0000","last_undegraded":"2024-09-19T09:30:50.155146+0000","last_fullsized":"2024-09-19T09:30:50.155146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:12:15.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563284+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:30:43.563284+0000","last_peered":"2024-09-19T09:30:43.563284+0000","last_clean":"2024-09-19T09:30:43.563284+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:30:43.563284+0000","last_undegraded":"2024-09-19T09:30:43.563284+0000","last_fullsized":"2024-09-19T09:30:43.563284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544223+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:30:43.544223+0000","last_peered":"2024-09-19T09:30:43.544223+0000","last_clean":"2024-09-19T09:30:43.544223+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:30:43.544223+0000","last_undegraded":"2024-09-19T09:30:43.544223+0000","last_fullsized":"2024-09-19T09:30:43.544223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188158+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:30:53.188158+0000","last_peered":"2024-09-19T09:30:53.188158+0000","last_clean":"2024-09-19T09:30:53.188158+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:30:53.188158+0000","last_undegraded":"2024-09-19T09:30:53.188158+0000","last_fullsized":"2024-09-19T09:30:53.188158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561983+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:30:43.561983+0000","last_peered":"2024-09-19T09:30:43.561983+0000","last_clean":"2024-09-19T09:30:43.561983+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:30:43.561983+0000","last_undegraded":"2024-09-19T09:30:43.561983+0000","last_fullsized":"2024-09-19T09:30:43.561983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192891+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:30:54.192891+0000","last_peered":"2024-09-19T09:30:54.192891+0000","last_clean":"2024-09-19T09:30:54.192891+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:30:54.192891+0000","last_undegraded":"2024-09-19T09:30:54.192891+0000","last_fullsized":"2024-09-19T09:30:54.192891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564534+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:30:43.564534+0000","last_peered":"2024-09-19T09:30:43.564534+0000","last_clean":"2024-09-19T09:30:43.564534+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:30:43.564534+0000","last_undegraded":"2024-09-19T09:30:43.564534+0000","last_fullsized":"2024-09-19T09:30:43.564534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.568190+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:30:43.568190+0000","last_peered":"2024-09-19T09:30:43.568190+0000","last_clean":"2024-09-19T09:30:43.568190+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:30:43.568190+0000","last_undegraded":"2024-09-19T09:30:43.568190+0000","last_fullsized":"2024-09-19T09:30:43.568190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543045+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:30:43.543045+0000","last_peered":"2024-09-19T09:30:43.543045+0000","last_clean":"2024-09-19T09:30:43.543045+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:30:43.543045+0000","last_undegraded":"2024-09-19T09:30:43.543045+0000","last_fullsized":"2024-09-19T09:30:43.543045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:02.206381+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:31:02.206381+0000","last_peered":"2024-09-19T09:31:02.206381+0000","last_clean":"2024-09-19T09:31:02.206381+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:31:02.206381+0000","last_undegraded":"2024-09-19T09:31:02.206381+0000","last_fullsized":"2024-09-19T09:31:02.206381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48666,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":227,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":155045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1047662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":892928,"data_stored":415105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":415483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1486848,"data_stored":1008789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1007681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":414530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":457413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":457193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:31:07.347 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:31:07.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:07 smithi028 ceph-mon[30972]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-19T09:31:07.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:07 smithi078 ceph-mon[35619]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-19T09:31:07.608 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:07.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:07 smithi038 ceph-mon[35271]: pgmap v289: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 -- 172.21.15.28:0/100231419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 msgr2=0x7fb6a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 --2- 172.21.15.28:0/100231419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41021c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009900 tx=0x7fb6a002a410 comp rx=0 tx=0).stop 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 -- 172.21.15.28:0/100231419 shutdown_connections 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 --2- 172.21.15.28:0/100231419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4102790 0x7fb6a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 --2- 172.21.15.28:0/100231419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4106f10 0x7fb6a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 --2- 172.21.15.28:0/100231419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41021c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 -- 172.21.15.28:0/100231419 >> 172.21.15.28:0/100231419 conn(0x7fb6a40fd8c0 msgr2=0x7fb6a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 -- 172.21.15.28:0/100231419 shutdown_connections 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.042+0000 7fb6abf46640 1 -- 172.21.15.28:0/100231419 wait complete. 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 Processor -- start 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 -- start start 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 0x7fb6a4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4106f10 0x7fb6a419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4112010 con 0x7fb6a4101de0 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4111e90 con 0x7fb6a4106f10 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6abf46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4112190 con 0x7fb6a4102790 2024-09-19T09:31:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56578/0 (socket says 172.21.15.28:56578) 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 -- 172.21.15.28:0/292808827 learned_addr learned my addr 172.21.15.28:0/292808827 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a94e1640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 0x7fb6a4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6aa4e3640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4106f10 0x7fb6a419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 -- 172.21.15.28:0/292808827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 msgr2=0x7fb6a4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 0x7fb6a4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 -- 172.21.15.28:0/292808827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4106f10 msgr2=0x7fb6a419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4106f10 0x7fb6a419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 -- 172.21.15.28:0/292808827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a419d560 con 0x7fb6a4101de0 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a94e1640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 0x7fb6a4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.043+0000 7fb6a9ce2640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005ab0 tx=0x7fb6a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.044+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a003f030 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.044+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a0003ce0 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.044+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a419d7f0 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.044+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0035070 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.044+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a41a5130 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.045+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a41032c0 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.046+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb6a0042030 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.046+0000 7fb69affd640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 0x7fb6900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.047+0000 7fb6a94e1640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 0x7fb6900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.047+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb6a00f0290 con 0x7fb6a4101de0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.047+0000 7fb6a94e1640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 0x7fb6900a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40ffde0 tx=0x7fb694008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.051+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb6a00bd000 con 0x7fb6a4101de0 2024-09-19T09:31:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.223+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6a4062630 con 0x7fb6900a32d0 2024-09-19T09:31:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.229+0000 7fb69affd640 1 -- 172.21.15.28:0/292808827 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258753 (secure 0 0 0) 0x7fb6a4062630 con 0x7fb6900a32d0 2024-09-19T09:31:08.233 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:31:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 msgr2=0x7fb6900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 0x7fb6900a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40ffde0 tx=0x7fb694008040 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 msgr2=0x7fb6a41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005ab0 tx=0x7fb6a00042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 shutdown_connections 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb6900a32d0 0x7fb6900a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a4106f10 0x7fb6a419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6a4102790 0x7fb6a4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 --2- 172.21.15.28:0/292808827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6a4101de0 0x7fb6a41985b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 >> 172.21.15.28:0/292808827 conn(0x7fb6a40fd8c0 msgr2=0x7fb6a40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 shutdown_connections 2024-09-19T09:31:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:08.232+0000 7fb6abf46640 1 -- 172.21.15.28:0/292808827 wait complete. 2024-09-19T09:31:08.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:08 smithi028 ceph-mon[30972]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:08.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:08 smithi078 ceph-mon[35619]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:08.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:08 smithi038 ceph-mon[35271]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:08.914 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-19T09:31:07.400858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639268,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":273,"num_read_kb":233,"num_write":410,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":337,"ondisk_log_size":337,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365696,"kb_used_data":12288,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576128,"statfs":{"total":1151940427776,"available":1151565955072,"internally_reserved":0,"allocated":12582912,"data_stored":6885600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42876,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":72,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565004+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:30:43.565004+0000","last_peered":"2024-09-19T09:30:43.565004+0000","last_clean":"2024-09-19T09:30:43.565004+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:30:43.565004+0000","last_undegraded":"2024-09-19T09:30:43.565004+0000","last_fullsized":"2024-09-19T09:30:43.565004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563865+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:30:43.563865+0000","last_peered":"2024-09-19T09:30:43.563865+0000","last_clean":"2024-09-19T09:30:43.563865+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:30:43.563865+0000","last_undegraded":"2024-09-19T09:30:43.563865+0000","last_fullsized":"2024-09-19T09:30:43.563865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564249+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:30:43.564249+0000","last_peered":"2024-09-19T09:30:43.564249+0000","last_clean":"2024-09-19T09:30:43.564249+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:30:43.564249+0000","last_undegraded":"2024-09-19T09:30:43.564249+0000","last_fullsized":"2024-09-19T09:30:43.564249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561198+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:30:43.561198+0000","last_peered":"2024-09-19T09:30:43.561198+0000","last_clean":"2024-09-19T09:30:43.561198+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:30:43.561198+0000","last_undegraded":"2024-09-19T09:30:43.561198+0000","last_fullsized":"2024-09-19T09:30:43.561198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561252+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:30:43.561252+0000","last_peered":"2024-09-19T09:30:43.561252+0000","last_clean":"2024-09-19T09:30:43.561252+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:30:43.561252+0000","last_undegraded":"2024-09-19T09:30:43.561252+0000","last_fullsized":"2024-09-19T09:30:43.561252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.547978+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:30:43.547978+0000","last_peered":"2024-09-19T09:30:43.547978+0000","last_clean":"2024-09-19T09:30:43.547978+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:30:43.547978+0000","last_undegraded":"2024-09-19T09:30:43.547978+0000","last_fullsized":"2024-09-19T09:30:43.547978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562172+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:30:43.562172+0000","last_peered":"2024-09-19T09:30:43.562172+0000","last_clean":"2024-09-19T09:30:43.562172+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:30:43.562172+0000","last_undegraded":"2024-09-19T09:30:43.562172+0000","last_fullsized":"2024-09-19T09:30:43.562172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542518+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:30:43.542518+0000","last_peered":"2024-09-19T09:30:43.542518+0000","last_clean":"2024-09-19T09:30:43.542518+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:30:43.542518+0000","last_undegraded":"2024-09-19T09:30:43.542518+0000","last_fullsized":"2024-09-19T09:30:43.542518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562779+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:30:43.562779+0000","last_peered":"2024-09-19T09:30:43.562779+0000","last_clean":"2024-09-19T09:30:43.562779+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:30:43.562779+0000","last_undegraded":"2024-09-19T09:30:43.562779+0000","last_fullsized":"2024-09-19T09:30:43.562779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548341+0000","last_change":"2024-09-19T09:30:42.564174+0000","last_active":"2024-09-19T09:30:43.548341+0000","last_peered":"2024-09-19T09:30:43.548341+0000","last_clean":"2024-09-19T09:30:43.548341+0000","last_became_active":"2024-09-19T09:30:42.563790+0000","last_became_peered":"2024-09-19T09:30:42.563790+0000","last_unstale":"2024-09-19T09:30:43.548341+0000","last_undegraded":"2024-09-19T09:30:43.548341+0000","last_fullsized":"2024-09-19T09:30:43.548341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:22.463896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564526+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:30:43.564526+0000","last_peered":"2024-09-19T09:30:43.564526+0000","last_clean":"2024-09-19T09:30:43.564526+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:30:43.564526+0000","last_undegraded":"2024-09-19T09:30:43.564526+0000","last_fullsized":"2024-09-19T09:30:43.564526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561947+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:30:43.561947+0000","last_peered":"2024-09-19T09:30:43.561947+0000","last_clean":"2024-09-19T09:30:43.561947+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:30:43.561947+0000","last_undegraded":"2024-09-19T09:30:43.561947+0000","last_fullsized":"2024-09-19T09:30:43.561947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561657+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:30:43.561657+0000","last_peered":"2024-09-19T09:30:43.561657+0000","last_clean":"2024-09-19T09:30:43.561657+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:30:43.561657+0000","last_undegraded":"2024-09-19T09:30:43.561657+0000","last_fullsized":"2024-09-19T09:30:43.561657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562189+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:30:43.562189+0000","last_peered":"2024-09-19T09:30:43.562189+0000","last_clean":"2024-09-19T09:30:43.562189+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:30:43.562189+0000","last_undegraded":"2024-09-19T09:30:43.562189+0000","last_fullsized":"2024-09-19T09:30:43.562189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562374+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:30:43.562374+0000","last_peered":"2024-09-19T09:30:43.562374+0000","last_clean":"2024-09-19T09:30:43.562374+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:30:43.562374+0000","last_undegraded":"2024-09-19T09:30:43.562374+0000","last_fullsized":"2024-09-19T09:30:43.562374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560910+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:30:43.560910+0000","last_peered":"2024-09-19T09:30:43.560910+0000","last_clean":"2024-09-19T09:30:43.560910+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:30:43.560910+0000","last_undegraded":"2024-09-19T09:30:43.560910+0000","last_fullsized":"2024-09-19T09:30:43.560910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565188+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:30:43.565188+0000","last_peered":"2024-09-19T09:30:43.565188+0000","last_clean":"2024-09-19T09:30:43.565188+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:30:43.565188+0000","last_undegraded":"2024-09-19T09:30:43.565188+0000","last_fullsized":"2024-09-19T09:30:43.565188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560190+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:30:43.560190+0000","last_peered":"2024-09-19T09:30:43.560190+0000","last_clean":"2024-09-19T09:30:43.560190+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:30:43.560190+0000","last_undegraded":"2024-09-19T09:30:43.560190+0000","last_fullsized":"2024-09-19T09:30:43.560190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563025+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:30:43.563025+0000","last_peered":"2024-09-19T09:30:43.563025+0000","last_clean":"2024-09-19T09:30:43.563025+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:30:43.563025+0000","last_undegraded":"2024-09-19T09:30:43.563025+0000","last_fullsized":"2024-09-19T09:30:43.563025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564597+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:30:43.564597+0000","last_peered":"2024-09-19T09:30:43.564597+0000","last_clean":"2024-09-19T09:30:43.564597+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:30:43.564597+0000","last_undegraded":"2024-09-19T09:30:43.564597+0000","last_fullsized":"2024-09-19T09:30:43.564597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561822+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:30:43.561822+0000","last_peered":"2024-09-19T09:30:43.561822+0000","last_clean":"2024-09-19T09:30:43.561822+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:30:43.561822+0000","last_undegraded":"2024-09-19T09:30:43.561822+0000","last_fullsized":"2024-09-19T09:30:43.561822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561430+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:30:43.561430+0000","last_peered":"2024-09-19T09:30:43.561430+0000","last_clean":"2024-09-19T09:30:43.561430+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:30:43.561430+0000","last_undegraded":"2024-09-19T09:30:43.561430+0000","last_fullsized":"2024-09-19T09:30:43.561430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542541+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:30:43.542541+0000","last_peered":"2024-09-19T09:30:43.542541+0000","last_clean":"2024-09-19T09:30:43.542541+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:30:43.542541+0000","last_undegraded":"2024-09-19T09:30:43.542541+0000","last_fullsized":"2024-09-19T09:30:43.542541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544194+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:30:43.544194+0000","last_peered":"2024-09-19T09:30:43.544194+0000","last_clean":"2024-09-19T09:30:43.544194+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:30:43.544194+0000","last_undegraded":"2024-09-19T09:30:43.544194+0000","last_fullsized":"2024-09-19T09:30:43.544194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542969+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:30:43.542969+0000","last_peered":"2024-09-19T09:30:43.542969+0000","last_clean":"2024-09-19T09:30:43.542969+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:30:43.542969+0000","last_undegraded":"2024-09-19T09:30:43.542969+0000","last_fullsized":"2024-09-19T09:30:43.542969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565943+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:30:43.565943+0000","last_peered":"2024-09-19T09:30:43.565943+0000","last_clean":"2024-09-19T09:30:43.565943+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:30:43.565943+0000","last_undegraded":"2024-09-19T09:30:43.565943+0000","last_fullsized":"2024-09-19T09:30:43.565943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564893+0000","last_change":"2024-09-19T09:30:42.559055+0000","last_active":"2024-09-19T09:30:43.564893+0000","last_peered":"2024-09-19T09:30:43.564893+0000","last_clean":"2024-09-19T09:30:43.564893+0000","last_became_active":"2024-09-19T09:30:42.558537+0000","last_became_peered":"2024-09-19T09:30:42.558537+0000","last_unstale":"2024-09-19T09:30:43.564893+0000","last_undegraded":"2024-09-19T09:30:43.564893+0000","last_fullsized":"2024-09-19T09:30:43.564893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.777026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562079+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:30:43.562079+0000","last_peered":"2024-09-19T09:30:43.562079+0000","last_clean":"2024-09-19T09:30:43.562079+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:30:43.562079+0000","last_undegraded":"2024-09-19T09:30:43.562079+0000","last_fullsized":"2024-09-19T09:30:43.562079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564360+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:30:43.564360+0000","last_peered":"2024-09-19T09:30:43.564360+0000","last_clean":"2024-09-19T09:30:43.564360+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:30:43.564360+0000","last_undegraded":"2024-09-19T09:30:43.564360+0000","last_fullsized":"2024-09-19T09:30:43.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562024+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:30:43.562024+0000","last_peered":"2024-09-19T09:30:43.562024+0000","last_clean":"2024-09-19T09:30:43.562024+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:30:43.562024+0000","last_undegraded":"2024-09-19T09:30:43.562024+0000","last_fullsized":"2024-09-19T09:30:43.562024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563172+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:30:43.563172+0000","last_peered":"2024-09-19T09:30:43.563172+0000","last_clean":"2024-09-19T09:30:43.563172+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:30:43.563172+0000","last_undegraded":"2024-09-19T09:30:43.563172+0000","last_fullsized":"2024-09-19T09:30:43.563172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563982+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:30:43.563982+0000","last_peered":"2024-09-19T09:30:43.563982+0000","last_clean":"2024-09-19T09:30:43.563982+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:30:43.563982+0000","last_undegraded":"2024-09-19T09:30:43.563982+0000","last_fullsized":"2024-09-19T09:30:43.563982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560987+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:30:43.560987+0000","last_peered":"2024-09-19T09:30:43.560987+0000","last_clean":"2024-09-19T09:30:43.560987+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:30:43.560987+0000","last_undegraded":"2024-09-19T09:30:43.560987+0000","last_fullsized":"2024-09-19T09:30:43.560987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:04.212005+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:31:04.212005+0000","last_peered":"2024-09-19T09:31:04.212005+0000","last_clean":"2024-09-19T09:31:04.212005+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:31:04.212005+0000","last_undegraded":"2024-09-19T09:31:04.212005+0000","last_fullsized":"2024-09-19T09:31:04.212005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544051+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:30:43.544051+0000","last_peered":"2024-09-19T09:30:43.544051+0000","last_clean":"2024-09-19T09:30:43.544051+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:30:43.544051+0000","last_undegraded":"2024-09-19T09:30:43.544051+0000","last_fullsized":"2024-09-19T09:30:43.544051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184065+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:30:52.184065+0000","last_peered":"2024-09-19T09:30:52.184065+0000","last_clean":"2024-09-19T09:30:52.184065+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:30:52.184065+0000","last_undegraded":"2024-09-19T09:30:52.184065+0000","last_fullsized":"2024-09-19T09:30:52.184065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.163943+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:30:51.163943+0000","last_peered":"2024-09-19T09:30:51.163943+0000","last_clean":"2024-09-19T09:30:51.163943+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:30:51.163943+0000","last_undegraded":"2024-09-19T09:30:51.163943+0000","last_fullsized":"2024-09-19T09:30:51.163943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542591+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:30:43.542591+0000","last_peered":"2024-09-19T09:30:43.542591+0000","last_clean":"2024-09-19T09:30:43.542591+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:30:43.542591+0000","last_undegraded":"2024-09-19T09:30:43.542591+0000","last_fullsized":"2024-09-19T09:30:43.542591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187883+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:30:53.187883+0000","last_peered":"2024-09-19T09:30:53.187883+0000","last_clean":"2024-09-19T09:30:53.187883+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:30:53.187883+0000","last_undegraded":"2024-09-19T09:30:53.187883+0000","last_fullsized":"2024-09-19T09:30:53.187883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.559954+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:30:43.559954+0000","last_peered":"2024-09-19T09:30:43.559954+0000","last_clean":"2024-09-19T09:30:43.559954+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:30:43.559954+0000","last_undegraded":"2024-09-19T09:30:43.559954+0000","last_fullsized":"2024-09-19T09:30:43.559954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.208238+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:31:03.208238+0000","last_peered":"2024-09-19T09:31:03.208238+0000","last_clean":"2024-09-19T09:31:03.208238+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:31:03.208238+0000","last_undegraded":"2024-09-19T09:31:03.208238+0000","last_fullsized":"2024-09-19T09:31:03.208238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562997+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:30:43.562997+0000","last_peered":"2024-09-19T09:30:43.562997+0000","last_clean":"2024-09-19T09:30:43.562997+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:30:43.562997+0000","last_undegraded":"2024-09-19T09:30:43.562997+0000","last_fullsized":"2024-09-19T09:30:43.562997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187517+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:30:53.187517+0000","last_peered":"2024-09-19T09:30:53.187517+0000","last_clean":"2024-09-19T09:30:53.187517+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:30:53.187517+0000","last_undegraded":"2024-09-19T09:30:53.187517+0000","last_fullsized":"2024-09-19T09:30:53.187517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564521+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:30:43.564521+0000","last_peered":"2024-09-19T09:30:43.564521+0000","last_clean":"2024-09-19T09:30:43.564521+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:30:43.564521+0000","last_undegraded":"2024-09-19T09:30:43.564521+0000","last_fullsized":"2024-09-19T09:30:43.564521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.182130+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:30:52.182130+0000","last_peered":"2024-09-19T09:30:52.182130+0000","last_clean":"2024-09-19T09:30:52.182130+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:30:52.182130+0000","last_undegraded":"2024-09-19T09:30:52.182130+0000","last_fullsized":"2024-09-19T09:30:52.182130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562134+0000","last_change":"2024-09-19T09:30:42.563614+0000","last_active":"2024-09-19T09:30:43.562134+0000","last_peered":"2024-09-19T09:30:43.562134+0000","last_clean":"2024-09-19T09:30:43.562134+0000","last_became_active":"2024-09-19T09:30:42.563289+0000","last_became_peered":"2024-09-19T09:30:42.563289+0000","last_unstale":"2024-09-19T09:30:43.562134+0000","last_undegraded":"2024-09-19T09:30:43.562134+0000","last_fullsized":"2024-09-19T09:30:43.562134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:47.215006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'23","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.212519+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:31:03.212519+0000","last_peered":"2024-09-19T09:31:03.212519+0000","last_clean":"2024-09-19T09:31:03.212519+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:31:03.212519+0000","last_undegraded":"2024-09-19T09:31:03.212519+0000","last_fullsized":"2024-09-19T09:31:03.212519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":52,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561453+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:30:43.561453+0000","last_peered":"2024-09-19T09:30:43.561453+0000","last_clean":"2024-09-19T09:30:43.561453+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:30:43.561453+0000","last_undegraded":"2024-09-19T09:30:43.561453+0000","last_fullsized":"2024-09-19T09:30:43.561453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.191692+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:30:54.191692+0000","last_peered":"2024-09-19T09:30:54.191692+0000","last_clean":"2024-09-19T09:30:54.191692+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:30:54.191692+0000","last_undegraded":"2024-09-19T09:30:54.191692+0000","last_fullsized":"2024-09-19T09:30:54.191692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561229+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:30:43.561229+0000","last_peered":"2024-09-19T09:30:43.561229+0000","last_clean":"2024-09-19T09:30:43.561229+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:30:43.561229+0000","last_undegraded":"2024-09-19T09:30:43.561229+0000","last_fullsized":"2024-09-19T09:30:43.561229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188935+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:30:53.188935+0000","last_peered":"2024-09-19T09:30:53.188935+0000","last_clean":"2024-09-19T09:30:53.188935+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:30:53.188935+0000","last_undegraded":"2024-09-19T09:30:53.188935+0000","last_fullsized":"2024-09-19T09:30:53.188935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564099+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:30:43.564099+0000","last_peered":"2024-09-19T09:30:43.564099+0000","last_clean":"2024-09-19T09:30:43.564099+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:30:43.564099+0000","last_undegraded":"2024-09-19T09:30:43.564099+0000","last_fullsized":"2024-09-19T09:30:43.564099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:05.212130+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:31:05.212130+0000","last_peered":"2024-09-19T09:31:05.212130+0000","last_clean":"2024-09-19T09:31:05.212130+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:31:05.212130+0000","last_undegraded":"2024-09-19T09:31:05.212130+0000","last_fullsized":"2024-09-19T09:31:05.212130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562555+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:30:43.562555+0000","last_peered":"2024-09-19T09:30:43.562555+0000","last_clean":"2024-09-19T09:30:43.562555+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:30:43.562555+0000","last_undegraded":"2024-09-19T09:30:43.562555+0000","last_fullsized":"2024-09-19T09:30:43.562555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186229+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:30:53.186229+0000","last_peered":"2024-09-19T09:30:53.186229+0000","last_clean":"2024-09-19T09:30:53.186229+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:30:53.186229+0000","last_undegraded":"2024-09-19T09:30:53.186229+0000","last_fullsized":"2024-09-19T09:30:53.186229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544131+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:30:43.544131+0000","last_peered":"2024-09-19T09:30:43.544131+0000","last_clean":"2024-09-19T09:30:43.544131+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:30:43.544131+0000","last_undegraded":"2024-09-19T09:30:43.544131+0000","last_fullsized":"2024-09-19T09:30:43.544131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.160939+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:30:53.160939+0000","last_peered":"2024-09-19T09:30:53.160939+0000","last_clean":"2024-09-19T09:30:53.160939+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:30:53.160939+0000","last_undegraded":"2024-09-19T09:30:53.160939+0000","last_fullsized":"2024-09-19T09:30:53.160939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564588+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:30:43.564588+0000","last_peered":"2024-09-19T09:30:43.564588+0000","last_clean":"2024-09-19T09:30:43.564588+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:30:43.564588+0000","last_undegraded":"2024-09-19T09:30:43.564588+0000","last_fullsized":"2024-09-19T09:30:43.564588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186972+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:30:53.186972+0000","last_peered":"2024-09-19T09:30:53.186972+0000","last_clean":"2024-09-19T09:30:53.186972+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:30:53.186972+0000","last_undegraded":"2024-09-19T09:30:53.186972+0000","last_fullsized":"2024-09-19T09:30:53.186972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564675+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:30:43.564675+0000","last_peered":"2024-09-19T09:30:43.564675+0000","last_clean":"2024-09-19T09:30:43.564675+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:30:43.564675+0000","last_undegraded":"2024-09-19T09:30:43.564675+0000","last_fullsized":"2024-09-19T09:30:43.564675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186823+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:30:53.186823+0000","last_peered":"2024-09-19T09:30:53.186823+0000","last_clean":"2024-09-19T09:30:53.186823+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:30:53.186823+0000","last_undegraded":"2024-09-19T09:30:53.186823+0000","last_fullsized":"2024-09-19T09:30:53.186823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564388+0000","last_change":"2024-09-19T09:30:42.558972+0000","last_active":"2024-09-19T09:30:43.564388+0000","last_peered":"2024-09-19T09:30:43.564388+0000","last_clean":"2024-09-19T09:30:43.564388+0000","last_became_active":"2024-09-19T09:30:42.558388+0000","last_became_peered":"2024-09-19T09:30:42.558388+0000","last_unstale":"2024-09-19T09:30:43.564388+0000","last_undegraded":"2024-09-19T09:30:43.564388+0000","last_fullsized":"2024-09-19T09:30:43.564388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:20.537069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:03.208965+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:31:03.208965+0000","last_peered":"2024-09-19T09:31:03.208965+0000","last_clean":"2024-09-19T09:31:03.208965+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:31:03.208965+0000","last_undegraded":"2024-09-19T09:31:03.208965+0000","last_fullsized":"2024-09-19T09:31:03.208965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564607+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:30:43.564607+0000","last_peered":"2024-09-19T09:30:43.564607+0000","last_clean":"2024-09-19T09:30:43.564607+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:30:43.564607+0000","last_undegraded":"2024-09-19T09:30:43.564607+0000","last_fullsized":"2024-09-19T09:30:43.564607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562904+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:30:43.562904+0000","last_peered":"2024-09-19T09:30:43.562904+0000","last_clean":"2024-09-19T09:30:43.562904+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:30:43.562904+0000","last_undegraded":"2024-09-19T09:30:43.562904+0000","last_fullsized":"2024-09-19T09:30:43.562904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192354+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:30:54.192354+0000","last_peered":"2024-09-19T09:30:54.192354+0000","last_clean":"2024-09-19T09:30:54.192354+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:30:54.192354+0000","last_undegraded":"2024-09-19T09:30:54.192354+0000","last_fullsized":"2024-09-19T09:30:54.192354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542792+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:30:43.542792+0000","last_peered":"2024-09-19T09:30:43.542792+0000","last_clean":"2024-09-19T09:30:43.542792+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:30:43.542792+0000","last_undegraded":"2024-09-19T09:30:43.542792+0000","last_fullsized":"2024-09-19T09:30:43.542792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.165255+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:30:51.165255+0000","last_peered":"2024-09-19T09:30:51.165255+0000","last_clean":"2024-09-19T09:30:51.165255+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:30:51.165255+0000","last_undegraded":"2024-09-19T09:30:51.165255+0000","last_fullsized":"2024-09-19T09:30:51.165255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561146+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:30:43.561146+0000","last_peered":"2024-09-19T09:30:43.561146+0000","last_clean":"2024-09-19T09:30:43.561146+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:30:43.561146+0000","last_undegraded":"2024-09-19T09:30:43.561146+0000","last_fullsized":"2024-09-19T09:30:43.561146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:01.207447+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:31:01.207447+0000","last_peered":"2024-09-19T09:31:01.207447+0000","last_clean":"2024-09-19T09:31:01.207447+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:31:01.207447+0000","last_undegraded":"2024-09-19T09:31:01.207447+0000","last_fullsized":"2024-09-19T09:31:01.207447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543968+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:30:43.543968+0000","last_peered":"2024-09-19T09:30:43.543968+0000","last_clean":"2024-09-19T09:30:43.543968+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:30:43.543968+0000","last_undegraded":"2024-09-19T09:30:43.543968+0000","last_fullsized":"2024-09-19T09:30:43.543968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:05.210583+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:31:05.210583+0000","last_peered":"2024-09-19T09:31:05.210583+0000","last_clean":"2024-09-19T09:31:05.210583+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:31:05.210583+0000","last_undegraded":"2024-09-19T09:31:05.210583+0000","last_fullsized":"2024-09-19T09:31:05.210583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548405+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:30:43.548405+0000","last_peered":"2024-09-19T09:30:43.548405+0000","last_clean":"2024-09-19T09:30:43.548405+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:30:43.548405+0000","last_undegraded":"2024-09-19T09:30:43.548405+0000","last_fullsized":"2024-09-19T09:30:43.548405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560204+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:30:43.560204+0000","last_peered":"2024-09-19T09:30:43.560204+0000","last_clean":"2024-09-19T09:30:43.560204+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:30:43.560204+0000","last_undegraded":"2024-09-19T09:30:43.560204+0000","last_fullsized":"2024-09-19T09:30:43.560204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:44.011551+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:30:44.011551+0000","last_peered":"2024-09-19T09:30:44.011551+0000","last_clean":"2024-09-19T09:30:44.011551+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:30:44.011551+0000","last_undegraded":"2024-09-19T09:30:44.011551+0000","last_fullsized":"2024-09-19T09:30:44.011551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188733+0000","last_change":"2024-09-19T09:30:40.592827+0000","last_active":"2024-09-19T09:30:53.188733+0000","last_peered":"2024-09-19T09:30:53.188733+0000","last_clean":"2024-09-19T09:30:53.188733+0000","last_became_active":"2024-09-19T09:30:40.591986+0000","last_became_peered":"2024-09-19T09:30:40.591986+0000","last_unstale":"2024-09-19T09:30:53.188733+0000","last_undegraded":"2024-09-19T09:30:53.188733+0000","last_fullsized":"2024-09-19T09:30:53.188733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:11.426185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562237+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:30:43.562237+0000","last_peered":"2024-09-19T09:30:43.562237+0000","last_clean":"2024-09-19T09:30:43.562237+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:30:43.562237+0000","last_undegraded":"2024-09-19T09:30:43.562237+0000","last_fullsized":"2024-09-19T09:30:43.562237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:57.560578+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:30:57.560578+0000","last_peered":"2024-09-19T09:30:57.560578+0000","last_clean":"2024-09-19T09:30:57.560578+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:30:57.560578+0000","last_undegraded":"2024-09-19T09:30:57.560578+0000","last_fullsized":"2024-09-19T09:30:57.560578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543123+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:30:43.543123+0000","last_peered":"2024-09-19T09:30:43.543123+0000","last_clean":"2024-09-19T09:30:43.543123+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:30:43.543123+0000","last_undegraded":"2024-09-19T09:30:43.543123+0000","last_fullsized":"2024-09-19T09:30:43.543123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186347+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:30:53.186347+0000","last_peered":"2024-09-19T09:30:53.186347+0000","last_clean":"2024-09-19T09:30:53.186347+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:30:53.186347+0000","last_undegraded":"2024-09-19T09:30:53.186347+0000","last_fullsized":"2024-09-19T09:30:53.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564238+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:30:43.564238+0000","last_peered":"2024-09-19T09:30:43.564238+0000","last_clean":"2024-09-19T09:30:43.564238+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:30:43.564238+0000","last_undegraded":"2024-09-19T09:30:43.564238+0000","last_fullsized":"2024-09-19T09:30:43.564238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155515+0000","last_change":"2024-09-19T09:30:40.592013+0000","last_active":"2024-09-19T09:30:50.155515+0000","last_peered":"2024-09-19T09:30:50.155515+0000","last_clean":"2024-09-19T09:30:50.155515+0000","last_became_active":"2024-09-19T09:30:40.591742+0000","last_became_peered":"2024-09-19T09:30:40.591742+0000","last_unstale":"2024-09-19T09:30:50.155515+0000","last_undegraded":"2024-09-19T09:30:50.155515+0000","last_fullsized":"2024-09-19T09:30:50.155515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:24.014799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560917+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:30:43.560917+0000","last_peered":"2024-09-19T09:30:43.560917+0000","last_clean":"2024-09-19T09:30:43.560917+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:30:43.560917+0000","last_undegraded":"2024-09-19T09:30:43.560917+0000","last_fullsized":"2024-09-19T09:30:43.560917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186945+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:30:53.186945+0000","last_peered":"2024-09-19T09:30:53.186945+0000","last_clean":"2024-09-19T09:30:53.186945+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:30:53.186945+0000","last_undegraded":"2024-09-19T09:30:53.186945+0000","last_fullsized":"2024-09-19T09:30:53.186945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564140+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:30:43.564140+0000","last_peered":"2024-09-19T09:30:43.564140+0000","last_clean":"2024-09-19T09:30:43.564140+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:30:43.564140+0000","last_undegraded":"2024-09-19T09:30:43.564140+0000","last_fullsized":"2024-09-19T09:30:43.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184164+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:30:52.184164+0000","last_peered":"2024-09-19T09:30:52.184164+0000","last_clean":"2024-09-19T09:30:52.184164+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:30:52.184164+0000","last_undegraded":"2024-09-19T09:30:52.184164+0000","last_fullsized":"2024-09-19T09:30:52.184164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542815+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:30:43.542815+0000","last_peered":"2024-09-19T09:30:43.542815+0000","last_clean":"2024-09-19T09:30:43.542815+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:30:43.542815+0000","last_undegraded":"2024-09-19T09:30:43.542815+0000","last_fullsized":"2024-09-19T09:30:43.542815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155146+0000","last_change":"2024-09-19T09:30:40.535163+0000","last_active":"2024-09-19T09:30:50.155146+0000","last_peered":"2024-09-19T09:30:50.155146+0000","last_clean":"2024-09-19T09:30:50.155146+0000","last_became_active":"2024-09-19T09:30:40.534756+0000","last_became_peered":"2024-09-19T09:30:40.534756+0000","last_unstale":"2024-09-19T09:30:50.155146+0000","last_undegraded":"2024-09-19T09:30:50.155146+0000","last_fullsized":"2024-09-19T09:30:50.155146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:12:15.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563284+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:30:43.563284+0000","last_peered":"2024-09-19T09:30:43.563284+0000","last_clean":"2024-09-19T09:30:43.563284+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:30:43.563284+0000","last_undegraded":"2024-09-19T09:30:43.563284+0000","last_fullsized":"2024-09-19T09:30:43.563284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544223+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:30:43.544223+0000","last_peered":"2024-09-19T09:30:43.544223+0000","last_clean":"2024-09-19T09:30:43.544223+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:30:43.544223+0000","last_undegraded":"2024-09-19T09:30:43.544223+0000","last_fullsized":"2024-09-19T09:30:43.544223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188158+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:30:53.188158+0000","last_peered":"2024-09-19T09:30:53.188158+0000","last_clean":"2024-09-19T09:30:53.188158+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:30:53.188158+0000","last_undegraded":"2024-09-19T09:30:53.188158+0000","last_fullsized":"2024-09-19T09:30:53.188158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561983+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:30:43.561983+0000","last_peered":"2024-09-19T09:30:43.561983+0000","last_clean":"2024-09-19T09:30:43.561983+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:30:43.561983+0000","last_undegraded":"2024-09-19T09:30:43.561983+0000","last_fullsized":"2024-09-19T09:30:43.561983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192891+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:30:54.192891+0000","last_peered":"2024-09-19T09:30:54.192891+0000","last_clean":"2024-09-19T09:30:54.192891+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:30:54.192891+0000","last_undegraded":"2024-09-19T09:30:54.192891+0000","last_fullsized":"2024-09-19T09:30:54.192891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564534+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:30:43.564534+0000","last_peered":"2024-09-19T09:30:43.564534+0000","last_clean":"2024-09-19T09:30:43.564534+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:30:43.564534+0000","last_undegraded":"2024-09-19T09:30:43.564534+0000","last_fullsized":"2024-09-19T09:30:43.564534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.568190+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:30:43.568190+0000","last_peered":"2024-09-19T09:30:43.568190+0000","last_clean":"2024-09-19T09:30:43.568190+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:30:43.568190+0000","last_undegraded":"2024-09-19T09:30:43.568190+0000","last_fullsized":"2024-09-19T09:30:43.568190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543045+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:30:43.543045+0000","last_peered":"2024-09-19T09:30:43.543045+0000","last_clean":"2024-09-19T09:30:43.543045+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:30:43.543045+0000","last_undegraded":"2024-09-19T09:30:43.543045+0000","last_fullsized":"2024-09-19T09:30:43.543045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:02.206381+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:31:02.206381+0000","last_peered":"2024-09-19T09:31:02.206381+0000","last_clean":"2024-09-19T09:31:02.206381+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:31:02.206381+0000","last_undegraded":"2024-09-19T09:31:02.206381+0000","last_fullsized":"2024-09-19T09:31:02.206381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48666,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":147,"num_read_kb":124,"num_write":227,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":155045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1047662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":892928,"data_stored":415105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":415483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1486848,"data_stored":1008789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1007681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":414647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":457413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":457193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:31:08.918 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:31:08.919 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T09:31:08.919 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T09:31:09.176 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:09.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:09 smithi028 ceph-mon[30972]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-19T09:31:09.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:09 smithi028 ceph-mon[30972]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:09 smithi078 ceph-mon[35619]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-19T09:31:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:09 smithi078 ceph-mon[35619]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:09.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:09 smithi038 ceph-mon[35271]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-19T09:31:09.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:09 smithi038 ceph-mon[35271]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 -- 172.21.15.28:0/2886179632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb8101c00 msgr2=0x7f0eb8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 --2- 172.21.15.28:0/2886179632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb8101c00 0x7f0eb8102000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009900 tx=0x7f0eac02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 -- 172.21.15.28:0/2886179632 shutdown_connections 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 --2- 172.21.15.28:0/2886179632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 0x7f0eb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 --2- 172.21.15.28:0/2886179632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb81037c0 0x7f0eb810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 --2- 172.21.15.28:0/2886179632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb8101c00 0x7f0eb8102000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 -- 172.21.15.28:0/2886179632 >> 172.21.15.28:0/2886179632 conn(0x7f0eb80fd390 msgr2=0x7f0eb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 -- 172.21.15.28:0/2886179632 shutdown_connections 2024-09-19T09:31:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.676+0000 7f0ebf049640 1 -- 172.21.15.28:0/2886179632 wait complete. 2024-09-19T09:31:09.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 Processor -- start 2024-09-19T09:31:09.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 -- start start 2024-09-19T09:31:09.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 0x7f0eb8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:09.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 0x7f0eb8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d8b0 con 0x7f0eb81037c0 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d730 con 0x7f0eb8102e00 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebf049640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810da30 con 0x7f0eb8101c00 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebd5e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebd5e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56596/0 (socket says 172.21.15.28:56596) 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0ebd5e6640 1 -- 172.21.15.28:0/416717508 learned_addr learned my addr 172.21.15.28:0/416717508 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.677+0000 7f0eb7fff640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 0x7f0eb8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebcde5640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 0x7f0eb8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 -- 172.21.15.28:0/416717508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 msgr2=0x7f0eb8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 0x7f0eb8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 -- 172.21.15.28:0/416717508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 msgr2=0x7f0eb8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 0x7f0eb8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 -- 172.21.15.28:0/416717508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb819fb30 con 0x7f0eb81037c0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebcde5640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 0x7f0eb8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebd5e6640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000eaa0 tx=0x7f0eb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000cdb0 con 0x7f0eb81037c0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0005020 con 0x7f0eb81037c0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00116b0 con 0x7f0eb81037c0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb819fe20 con 0x7f0eb81037c0 2024-09-19T09:31:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.678+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a0270 con 0x7f0eb81037c0 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.679+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb8108fb0 con 0x7f0eb81037c0 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.680+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0eb0005560 con 0x7f0eb81037c0 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.681+0000 7f0eb5ffb640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 0x7f0ea80a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.681+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0eb00cfaa0 con 0x7f0eb81037c0 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.681+0000 7f0ebcde5640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 0x7f0ea80a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.682+0000 7f0ebcde5640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 0x7f0ea80a5840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02aa00 tx=0x7f0eac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.685+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0eb0016030 con 0x7f0eb81037c0 2024-09-19T09:31:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.928+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0eb8062df0 con 0x7f0eb81037c0 2024-09-19T09:31:09.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.930+0000 7f0eb5ffb640 1 -- 172.21.15.28:0/416717508 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f0eb009c810 con 0x7f0eb81037c0 2024-09-19T09:31:09.932 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-19T09:31:09.932 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBd7+tmVpNsNxAAWl5L1OtLkITVlCK2ky3jbw== 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 msgr2=0x7f0ea80a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 0x7f0ea80a5840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02aa00 tx=0x7f0eac000bc0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 msgr2=0x7f0eb819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000eaa0 tx=0x7f0eb000ef60 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 shutdown_connections 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ea80a3380 0x7f0ea80a5840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8102e00 0x7f0eb8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0eb81037c0 0x7f0eb819f4c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 --2- 172.21.15.28:0/416717508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb8101c00 0x7f0eb8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 >> 172.21.15.28:0/416717508 conn(0x7f0eb80fd390 msgr2=0x7f0eb8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 shutdown_connections 2024-09-19T09:31:09.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:09.933+0000 7f0ebf049640 1 -- 172.21.15.28:0/416717508 wait complete. 2024-09-19T09:31:10.485 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:31:10.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T09:31:10.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T09:31:10.507 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:10 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:31:10.508 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:10 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:31:10.522 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T09:31:10.522 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T09:31:10.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T09:31:10.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:10 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:31:10.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:10 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:31:10.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:10 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:31:10.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:10 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/416717508' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:31:10.819 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:11.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 -- 172.21.15.28:0/188190094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 msgr2=0x7fb4a8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:11.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 --2- 172.21.15.28:0/188190094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8103120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb49802a480 comp rx=0 tx=0).stop 2024-09-19T09:31:11.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 -- 172.21.15.28:0/188190094 shutdown_connections 2024-09-19T09:31:11.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 --2- 172.21.15.28:0/188190094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 --2- 172.21.15.28:0/188190094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 0x7fb4a810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 --2- 172.21.15.28:0/188190094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8103120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 -- 172.21.15.28:0/188190094 >> 172.21.15.28:0/188190094 conn(0x7fb4a80fc7a0 msgr2=0x7fb4a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 -- 172.21.15.28:0/188190094 shutdown_connections 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.242+0000 7fb4acea9640 1 -- 172.21.15.28:0/188190094 wait complete. 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 Processor -- start 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 -- start start 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 0x7fb4a8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8111fc0 con 0x7fb4a8102d40 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8111e40 con 0x7fb4a810c170 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4acea9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112140 con 0x7fb4a8103660 2024-09-19T09:31:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56614/0 (socket says 172.21.15.28:56614) 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 -- 172.21.15.28:0/1836481354 learned_addr learned my addr 172.21.15.28:0/1836481354 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a5d9b640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 0x7fb4a8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a6d9d640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 -- 172.21.15.28:0/1836481354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 msgr2=0x7fb4a8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 0x7fb4a8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 -- 172.21.15.28:0/1836481354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 msgr2=0x7fb4a819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a659c640 1 -- 172.21.15.28:0/1836481354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a819d4b0 con 0x7fb4a8102d40 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.243+0000 7fb4a6d9d640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4a659c640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb498004770 tx=0x7fb498003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49803f070 con 0x7fb4a8102d40 2024-09-19T09:31:11.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a819d740 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498031ca0 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498035030 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.244+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a5080 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.245+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a8103120 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.246+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb498042070 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.247+0000 7fb4977fe640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 0x7fb4900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.247+0000 7fb4a5d9b640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 0x7fb4900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.247+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb4980f0020 con 0x7fb4a8102d40 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.247+0000 7fb4a5d9b640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 0x7fb4900a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb4a810d5d0 tx=0x7fb49c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.251+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4980bcd90 con 0x7fb4a8102d40 2024-09-19T09:31:11.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.475+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb4a81071e0 con 0x7fb4a8102d40 2024-09-19T09:31:11.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.480+0000 7fb4977fe640 1 -- 172.21.15.28:0/1836481354 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264043 (secure 0 0 0) 0x7fb4980bc690 con 0x7fb4a8102d40 2024-09-19T09:31:11.485 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 msgr2=0x7fb4900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 0x7fb4900a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb4a810d5d0 tx=0x7fb49c005f90 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 msgr2=0x7fb4a8198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb498004770 tx=0x7fb498003c10 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 shutdown_connections 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb4900a32d0 0x7fb4900a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4a810c170 0x7fb4a819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8103660 0x7fb4a8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 --2- 172.21.15.28:0/1836481354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4a8102d40 0x7fb4a8198500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.482+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 >> 172.21.15.28:0/1836481354 conn(0x7fb4a80fc7a0 msgr2=0x7fb4a80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.483+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 shutdown_connections 2024-09-19T09:31:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:11.483+0000 7fb4acea9640 1 -- 172.21.15.28:0/1836481354 wait complete. 2024-09-19T09:31:11.494 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:11 smithi028 ceph-mon[30972]: pgmap v291: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-19T09:31:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:11 smithi078 ceph-mon[35619]: pgmap v291: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-19T09:31:11.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:11 smithi038 ceph-mon[35271]: pgmap v291: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-19T09:31:12.118 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":147346946},{"type":"v1","addr":"172.21.15.28:6801","nonce":147346946}]},"active_addr":"172.21.15.28:6801/147346946","active_change":"2024-09-19T09:23:47.303458+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.28:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":4252722596}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":193990680}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":21720899}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":390332578}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1538281561}]}]} 2024-09-19T09:31:12.125 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T09:31:12.125 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T09:31:12.125 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:31:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:12.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:12 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1836481354' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:31:12.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:12 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1836481354' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:31:12.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:12 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1836481354' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:31:12.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 -- 172.21.15.28:0/957065222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c0ffb60 msgr2=0x7f799c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:12.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 --2- 172.21.15.28:0/957065222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c0ffb60 0x7f799c0fffc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7984009900 tx=0x7f798402a410 comp rx=0 tx=0).stop 2024-09-19T09:31:12.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 -- 172.21.15.28:0/957065222 shutdown_connections 2024-09-19T09:31:12.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 --2- 172.21.15.28:0/957065222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c100500 0x7f799c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:12.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 --2- 172.21.15.28:0/957065222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c0ffb60 0x7f799c0fffc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:12.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 --2- 172.21.15.28:0/957065222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c106b80 0x7f799c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:12.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.839+0000 7f79a1371640 1 -- 172.21.15.28:0/957065222 >> 172.21.15.28:0/957065222 conn(0x7f799c0fba90 msgr2=0x7f799c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:12.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- 172.21.15.28:0/957065222 shutdown_connections 2024-09-19T09:31:12.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- 172.21.15.28:0/957065222 wait complete. 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 Processor -- start 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- start start 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 0x7f799c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 0x7f799c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f799c111f60 con 0x7f799c100500 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f799c111de0 con 0x7f799c106b80 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.840+0000 7f79a1371640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1120e0 con 0x7f799c0ffb60 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:12.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56640/0 (socket says 172.21.15.28:56640) 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 -- 172.21.15.28:0/4026355277 learned_addr learned my addr 172.21.15.28:0/4026355277 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f799affd640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 0x7f799c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f799b7fe640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 0x7f799c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 -- 172.21.15.28:0/4026355277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 msgr2=0x7f799c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 0x7f799c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 -- 172.21.15.28:0/4026355277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 msgr2=0x7f799c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 0x7f799c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 -- 172.21.15.28:0/4026355277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c19d3f0 con 0x7f799c100500 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f799affd640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 0x7f799c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f799b7fe640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 0x7f799c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7993fff640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f7984003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798403f070 con 0x7f799c100500 2024-09-19T09:31:12.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7984031c30 con 0x7f799c100500 2024-09-19T09:31:12.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c19d680 con 0x7f799c100500 2024-09-19T09:31:12.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984035030 con 0x7f799c100500 2024-09-19T09:31:12.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.841+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a4fc0 con 0x7f799c100500 2024-09-19T09:31:12.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.842+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c101080 con 0x7f799c100500 2024-09-19T09:31:12.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.844+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7984042070 con 0x7f799c100500 2024-09-19T09:31:12.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.845+0000 7f7998ff9640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 0x7f79880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:12.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.845+0000 7f799affd640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 0x7f79880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:12.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.845+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f79840f0580 con 0x7f799c100500 2024-09-19T09:31:12.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.845+0000 7f799affd640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 0x7f79880a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f798c005d00 tx=0x7f798c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:12.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:12.848+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79840bd2f0 con 0x7f799c100500 2024-09-19T09:31:13.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.028+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f799c10a0e0 con 0x7f799c100500 2024-09-19T09:31:13.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.029+0000 7f7998ff9640 1 -- 172.21.15.28:0/4026355277 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22195 (secure 0 0 0) 0x7f79840bcbf0 con 0x7f799c100500 2024-09-19T09:31:13.031 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:13.031 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":80,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:30:43.054505+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.031+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 msgr2=0x7f79880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.031+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 0x7f79880a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f798c005d00 tx=0x7f798c009550 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.031+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 msgr2=0x7f799c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.031+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7984006010 tx=0x7f7984003c10 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 shutdown_connections 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79880a32d0 0x7f79880a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799c106b80 0x7f799c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f799c100500 0x7f799c198980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 --2- 172.21.15.28:0/4026355277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f799c0ffb60 0x7f799c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 >> 172.21.15.28:0/4026355277 conn(0x7f799c0fba90 msgr2=0x7f799c0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:13.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 shutdown_connections 2024-09-19T09:31:13.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:13.032+0000 7f79a1371640 1 -- 172.21.15.28:0/4026355277 wait complete. 2024-09-19T09:31:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:13 smithi028 ceph-mon[30972]: pgmap v292: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T09:31:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4026355277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:13 smithi078 ceph-mon[35619]: pgmap v292: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T09:31:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4026355277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:13.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:13 smithi038 ceph-mon[35271]: pgmap v292: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T09:31:13.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4026355277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:13.793 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T09:31:13.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:31:14.055 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:14.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 -- 172.21.15.28:0/758936847 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 msgr2=0x7f1fac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:14.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 --2- 172.21.15.28:0/758936847 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac10ea90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009ca0 tx=0x7f1fa402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 -- 172.21.15.28:0/758936847 shutdown_connections 2024-09-19T09:31:14.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 --2- 172.21.15.28:0/758936847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac102790 0x7f1fac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 --2- 172.21.15.28:0/758936847 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac10ea90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 --2- 172.21.15.28:0/758936847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac101de0 0x7f1fac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 -- 172.21.15.28:0/758936847 >> 172.21.15.28:0/758936847 conn(0x7f1fac0fd8c0 msgr2=0x7f1fac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 -- 172.21.15.28:0/758936847 shutdown_connections 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.497+0000 7f1fb1796640 1 -- 172.21.15.28:0/758936847 wait complete. 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 Processor -- start 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 -- start start 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac101de0 0x7f1fac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac102790 0x7f1fac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac111f70 con 0x7f1fac106f10 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac111df0 con 0x7f1fac101de0 2024-09-19T09:31:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fb1796640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1120f0 con 0x7f1fac102790 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fab7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fab7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42214/0 (socket says 172.21.15.28:42214) 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fab7fe640 1 -- 172.21.15.28:0/4252551170 learned_addr learned my addr 172.21.15.28:0/4252551170 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1faaffd640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac101de0 0x7f1fac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.498+0000 7f1fa27fc640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac102790 0x7f1fac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 -- 172.21.15.28:0/4252551170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac102790 msgr2=0x7f1fac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac102790 0x7f1fac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 -- 172.21.15.28:0/4252551170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac101de0 msgr2=0x7f1fac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac101de0 0x7f1fac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 -- 172.21.15.28:0/4252551170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac1a6890 con 0x7f1fac106f10 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fab7fe640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4031a20 tx=0x7f1fa4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4003c30 con 0x7f1fac106f10 2024-09-19T09:31:14.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac1a6b20 con 0x7f1fac106f10 2024-09-19T09:31:14.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa4003dd0 con 0x7f1fac106f10 2024-09-19T09:31:14.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac1a7060 con 0x7f1fac106f10 2024-09-19T09:31:14.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.499+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa402fc90 con 0x7f1fac106f10 2024-09-19T09:31:14.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.501+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fac1021c0 con 0x7f1fac106f10 2024-09-19T09:31:14.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.501+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1fa4004000 con 0x7f1fac106f10 2024-09-19T09:31:14.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.502+0000 7f1fa8ff9640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 0x7f1f980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:14.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.502+0000 7f1faaffd640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 0x7f1f980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:14.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.502+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1fa40f0c60 con 0x7f1fac106f10 2024-09-19T09:31:14.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.503+0000 7f1faaffd640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 0x7f1f980a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005fd0 tx=0x7f1f9c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:14.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.506+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1fa40bdbd0 con 0x7f1fac106f10 2024-09-19T09:31:14.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.687+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fac10a150 con 0x7f1fac106f10 2024-09-19T09:31:14.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.688+0000 7f1fa8ff9640 1 -- 172.21.15.28:0/4252551170 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22195 (secure 0 0 0) 0x7f1fa40bd4d0 con 0x7f1fac106f10 2024-09-19T09:31:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:14.691 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":80,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:30:43.054505+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:31:14.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 msgr2=0x7f1f980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 0x7f1f980a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005fd0 tx=0x7f1f9c00b390 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 msgr2=0x7f1fac074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4031a20 tx=0x7f1fa4031a50 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 shutdown_connections 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1f980a32d0 0x7f1f980a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fac106f10 0x7f1fac074f70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fac102790 0x7f1fac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 --2- 172.21.15.28:0/4252551170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fac101de0 0x7f1fac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 >> 172.21.15.28:0/4252551170 conn(0x7f1fac0fd8c0 msgr2=0x7f1fac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 shutdown_connections 2024-09-19T09:31:14.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:14.691+0000 7f1fb1796640 1 -- 172.21.15.28:0/4252551170 wait complete. 2024-09-19T09:31:15.388 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:15 smithi028 ceph-mon[30972]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-09-19T09:31:15.389 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:15 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4252551170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:15.419 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T09:31:15.420 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T09:31:15.420 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T09:31:15.421 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T09:31:15.421 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T09:31:15.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T09:31:15.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T09:31:15.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T09:31:15.423 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T09:31:15.423 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T09:31:15.424 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T09:31:15.424 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T09:31:15.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:15 smithi078 ceph-mon[35619]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-09-19T09:31:15.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:15 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4252551170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:15.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:15 smithi038 ceph-mon[35271]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-09-19T09:31:15.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:15 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4252551170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:16.156 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.158 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.211 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.213 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.218 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.218 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:17.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 -- 172.21.15.28:0/741770011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 msgr2=0x7f7dec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/741770011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec1029b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009900 tx=0x7f7ddc02a480 comp rx=0 tx=0).stop 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 -- 172.21.15.28:0/741770011 shutdown_connections 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/741770011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 0x7f7dec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/741770011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec1029b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/741770011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100e20 0x7f7dec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 -- 172.21.15.28:0/741770011 >> 172.21.15.28:0/741770011 conn(0x7f7dec0fc7d0 msgr2=0x7f7dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 -- 172.21.15.28:0/741770011 shutdown_connections 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.135+0000 7f7df3fbe640 1 -- 172.21.15.28:0/741770011 wait complete. 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 Processor -- start 2024-09-19T09:31:17.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 -- start start 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100e20 0x7f7dec198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 0x7f7dec19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec111020 con 0x7f7dec102570 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec110ea0 con 0x7f7dec102ef0 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df3fbe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec1111a0 con 0x7f7dec100e20 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df1559640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:17.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df1559640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42230/0 (socket says 172.21.15.28:42230) 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df1559640 1 -- 172.21.15.28:0/4057568087 learned_addr learned my addr 172.21.15.28:0/4057568087 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df1d5a640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100e20 0x7f7dec198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.136+0000 7f7df255b640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 0x7f7dec19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 -- 172.21.15.28:0/4057568087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100e20 msgr2=0x7f7dec198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec100e20 0x7f7dec198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 -- 172.21.15.28:0/4057568087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 msgr2=0x7f7dec19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 0x7f7dec19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19d620 con 0x7f7dec102570 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df255b640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dec102ef0 0x7f7dec19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:17.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df1559640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec198bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc03f070 con 0x7f7dec102570 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ddc031e30 con 0x7f7dec102570 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19d8b0 con 0x7f7dec102570 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a51f0 con 0x7f7dec102570 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.137+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc035030 con 0x7f7dec102570 2024-09-19T09:31:17.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.138+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7dd0001030 con 0x7f7dec102570 2024-09-19T09:31:17.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.139+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7ddc042070 con 0x7f7dec102570 2024-09-19T09:31:17.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7de2ffd640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a3350 0x7f7dd80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:17.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7df1d5a640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a3350 0x7f7dd80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:17.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7ddc0f09a0 con 0x7f7dec102570 2024-09-19T09:31:17.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7de2ffd640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] conn(0x7f7dd80acba0 0x7f7dd80aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ddc038e90 con 0x7f7dd80acba0 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7ddc0f0d50 con 0x7f7dec102570 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7df255b640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] conn(0x7f7dd80acba0 0x7f7dd80aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.140+0000 7f7df1d5a640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a3350 0x7f7dd80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009990 tx=0x7f7de8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.141+0000 7f7df255b640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] conn(0x7f7dd80acba0 0x7f7dd80aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:17.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.141+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== osd.0 v2:172.21.15.28:6802/1911211878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ddc038e90 con 0x7f7dd80acba0 2024-09-19T09:31:17.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.155+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 --> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7dd0002dd0 con 0x7f7dd80acba0 2024-09-19T09:31:17.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.155+0000 7f7de2ffd640 1 -- 172.21.15.28:0/4057568087 <== osd.0 v2:172.21.15.28:6802/1911211878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7dd0002dd0 con 0x7f7dd80acba0 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.155+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] conn(0x7f7dd80acba0 msgr2=0x7f7dd80aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] conn(0x7f7dd80acba0 0x7f7dd80aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a3350 msgr2=0x7f7dd80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dd80a3350 0x7f7dd80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009990 tx=0x7f7de8009110 comp rx=0 tx=0).stop 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 msgr2=0x7f7dec198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 --2- 172.21.15.28:0/4057568087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dec102570 0x7f7dec198bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc003c80 comp rx=0 tx=0).stop 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df255b640 1 -- 172.21.15.28:0/4057568087 reap_dead start 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 shutdown_connections 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 >> 172.21.15.28:0/4057568087 conn(0x7f7dec0fc7d0 msgr2=0x7f7dec06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:17.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 shutdown_connections 2024-09-19T09:31:17.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:17.156+0000 7f7df3fbe640 1 -- 172.21.15.28:0/4057568087 wait complete. 2024-09-19T09:31:17.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:17 smithi028 ceph-mon[30972]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T09:31:17.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:17 smithi078 ceph-mon[35619]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T09:31:17.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:17 smithi038 ceph-mon[35271]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T09:31:18.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 -- 172.21.15.28:0/2054868003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464101de0 msgr2=0x7fc4641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 --2- 172.21.15.28:0/2054868003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464101de0 0x7fc4641021c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc454009900 tx=0x7fc45402a410 comp rx=0 tx=0).stop 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 -- 172.21.15.28:0/2054868003 shutdown_connections 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 --2- 172.21.15.28:0/2054868003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464102790 0x7fc4641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 --2- 172.21.15.28:0/2054868003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc464106f10 0x7fc46410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.130+0000 7fc468937640 1 --2- 172.21.15.28:0/2054868003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464101de0 0x7fc4641021c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.131+0000 7fc468937640 1 -- 172.21.15.28:0/2054868003 >> 172.21.15.28:0/2054868003 conn(0x7fc4640fd8c0 msgr2=0x7fc4640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.131+0000 7fc468937640 1 -- 172.21.15.28:0/2054868003 shutdown_connections 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.131+0000 7fc468937640 1 -- 172.21.15.28:0/2054868003 wait complete. 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.131+0000 7fc468937640 1 Processor -- start 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.131+0000 7fc468937640 1 -- start start 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc464101de0 0x7fc464198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 0x7fc464198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464106f10 0x7fc46419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc464111f70 con 0x7fc464102790 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc464111df0 con 0x7fc464106f10 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641120f0 con 0x7fc464101de0 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 0x7fc464198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 0x7fc464198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42240/0 (socket says 172.21.15.28:42240) 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 -- 172.21.15.28:0/302794494 learned_addr learned my addr 172.21.15.28:0/302794494 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc462d9d640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464106f10 0x7fc46419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc46259c640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc464101de0 0x7fc464198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 -- 172.21.15.28:0/302794494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc464101de0 msgr2=0x7fc464198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc464101de0 0x7fc464198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 -- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464106f10 msgr2=0x7fc46419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc464106f10 0x7fc46419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46419d3f0 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc461d9b640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 0x7fc464198980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d950 tx=0x7fc45800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458017070 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46419d6e0 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc458009c30 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45800f6b0 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.132+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a4fc0 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.133+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc464062980 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.135+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc4580109d0 con 0x7fc464102790 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.135+0000 7fc4537fe640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc44c0a32d0 0x7fc44c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc46259c640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc44c0a32d0 0x7fc44c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc4580cf980 con 0x7fc464102790 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc4537fe640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] conn(0x7fc44c0acb20 0x7fc44c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc44c0af5f0 con 0x7fc44c0acb20 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fc4580cfe60 con 0x7fc464102790 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc462d9d640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] conn(0x7fc44c0acb20 0x7fc44c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc46259c640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc44c0a32d0 0x7fc44c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc454005ab0 tx=0x7fc454005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.136+0000 7fc462d9d640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] conn(0x7fc44c0acb20 0x7fc44c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.137+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== osd.7 v2:172.21.15.38:6816/917618380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc44c0af5f0 con 0x7fc44c0acb20 2024-09-19T09:31:18.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.151+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 --> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc464062d80 con 0x7fc44c0acb20 2024-09-19T09:31:18.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc4537fe640 1 -- 172.21.15.28:0/302794494 <== osd.7 v2:172.21.15.38:6816/917618380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc464062d80 con 0x7fc44c0acb20 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] conn(0x7fc44c0acb20 msgr2=0x7fc44c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] conn(0x7fc44c0acb20 0x7fc44c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc44c0a32d0 msgr2=0x7fc44c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc44c0a32d0 0x7fc44c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc454005ab0 tx=0x7fc454005a20 comp rx=0 tx=0).stop 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 msgr2=0x7fc464198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 --2- 172.21.15.28:0/302794494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc464102790 0x7fc464198980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d950 tx=0x7fc45800de10 comp rx=0 tx=0).stop 2024-09-19T09:31:18.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc462d9d640 1 -- 172.21.15.28:0/302794494 reap_dead start 2024-09-19T09:31:18.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 shutdown_connections 2024-09-19T09:31:18.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 >> 172.21.15.28:0/302794494 conn(0x7fc4640fd8c0 msgr2=0x7fc4640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:18.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 shutdown_connections 2024-09-19T09:31:18.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.152+0000 7fc468937640 1 -- 172.21.15.28:0/302794494 wait complete. 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 -- 172.21.15.28:0/1173942341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 msgr2=0x7f0124106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 --2- 172.21.15.28:0/1173942341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f0124106e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0114009930 tx=0x7f011402a420 comp rx=0 tx=0).stop 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 -- 172.21.15.28:0/1173942341 shutdown_connections 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 --2- 172.21.15.28:0/1173942341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0124100a80 0x7f0124100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 --2- 172.21.15.28:0/1173942341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0124101420 0x7f012410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 --2- 172.21.15.28:0/1173942341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f0124106e60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 -- 172.21.15.28:0/1173942341 >> 172.21.15.28:0/1173942341 conn(0x7f01240fc7a0 msgr2=0x7f01240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:18.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.340+0000 7f012a300640 1 -- 172.21.15.28:0/1173942341 shutdown_connections 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 -- 172.21.15.28:0/1173942341 wait complete. 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 Processor -- start 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 -- start start 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0124100a80 0x7f0124197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0124101420 0x7f0124197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.341+0000 7f012a300640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f012419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012a300640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0124110ea0 con 0x7f0124106a80 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012a300640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0124110d20 con 0x7f0124101420 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012a300640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0124111020 con 0x7f0124100a80 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f012419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f012419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42254/0 (socket says 172.21.15.28:42254) 2024-09-19T09:31:18.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 -- 172.21.15.28:0/544694937 learned_addr learned my addr 172.21.15.28:0/544694937 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f01237fe640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0124101420 0x7f0124197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f0123fff640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0124100a80 0x7f0124197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 -- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0124100a80 msgr2=0x7f0124197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0124100a80 0x7f0124197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 -- 172.21.15.28:0/544694937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0124101420 msgr2=0x7f0124197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0124101420 0x7f0124197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012419c2b0 con 0x7f0124106a80 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012889d640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f012419bbd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b970 tx=0x7f011c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c019070 con 0x7f0124106a80 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c014440 con 0x7f0124106a80 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012419c5a0 con 0x7f0124106a80 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c0075d0 con 0x7f0124106a80 2024-09-19T09:31:18.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.342+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01241a3e80 con 0x7f0124106a80 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.343+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01240fd820 con 0x7f0124106a80 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.345+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f011c01c070 con 0x7f0124106a80 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.345+0000 7f01217fa640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f010c0a32d0 0x7f010c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f0123fff640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f010c0a32d0 0x7f010c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f011c0cf6c0 con 0x7f0124106a80 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01217fa640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] conn(0x7f010c0acb20 0x7f010c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f010c0af5f0 con 0x7f010c0acb20 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f011c0cfa70 con 0x7f0124106a80 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f0123fff640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f010c0a32d0 0x7f010c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f0114031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01237fe640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] conn(0x7f010c0acb20 0x7f010c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.346+0000 7f01237fe640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] conn(0x7f010c0acb20 0x7f010c0aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:18.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.347+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== osd.5 v2:172.21.15.78:6808/3861457216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f010c0af5f0 con 0x7f010c0acb20 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.362+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 --> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01241021c0 con 0x7f010c0acb20 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.362+0000 7f01217fa640 1 -- 172.21.15.28:0/544694937 <== osd.5 v2:172.21.15.78:6808/3861457216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f01241021c0 con 0x7f010c0acb20 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] conn(0x7f010c0acb20 msgr2=0x7f010c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] conn(0x7f010c0acb20 0x7f010c0aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f010c0a32d0 msgr2=0x7f010c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f010c0a32d0 0x7f010c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f0114031490 comp rx=0 tx=0).stop 2024-09-19T09:31:18.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 msgr2=0x7f012419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 --2- 172.21.15.28:0/544694937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0124106a80 0x7f012419bbd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b970 tx=0x7f011c00bd70 comp rx=0 tx=0).stop 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012889d640 1 -- 172.21.15.28:0/544694937 reap_dead start 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 shutdown_connections 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 >> 172.21.15.28:0/544694937 conn(0x7f01240fc7a0 msgr2=0x7f01240fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 shutdown_connections 2024-09-19T09:31:18.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:18.363+0000 7f012a300640 1 -- 172.21.15.28:0/544694937 wait complete. 2024-09-19T09:31:19.216 INFO:teuthology.orchestra.run.smithi028.stdout:158913790002 2024-09-19T09:31:19.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T09:31:19.493 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:19 smithi028 ceph-mon[30972]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T09:31:19.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:19 smithi078 ceph-mon[35619]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T09:31:19.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:19 smithi038 ceph-mon[35271]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T09:31:19.870 INFO:teuthology.orchestra.run.smithi028.stdout:210453397544 2024-09-19T09:31:19.871 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- 172.21.15.28:0/1125358538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8100e20 msgr2=0x7efdc8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 --2- 172.21.15.28:0/1125358538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8100e20 0x7efdc8101200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009900 tx=0x7efdbc02a480 comp rx=0 tx=0).stop 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- 172.21.15.28:0/1125358538 shutdown_connections 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 --2- 172.21.15.28:0/1125358538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc8102ef0 0x7efdc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 --2- 172.21.15.28:0/1125358538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 0x7efdc81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 --2- 172.21.15.28:0/1125358538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8100e20 0x7efdc8101200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- 172.21.15.28:0/1125358538 >> 172.21.15.28:0/1125358538 conn(0x7efdc80fc7d0 msgr2=0x7efdc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- 172.21.15.28:0/1125358538 shutdown_connections 2024-09-19T09:31:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- 172.21.15.28:0/1125358538 wait complete. 2024-09-19T09:31:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 Processor -- start 2024-09-19T09:31:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.035+0000 7efdcee5a640 1 -- start start 2024-09-19T09:31:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc8100e20 0x7efdc8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 0x7efdc8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 0x7efdc819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8110f80 con 0x7efdc8102ef0 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8110e00 con 0x7efdc8100e20 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcee5a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8111100 con 0x7efdc8102570 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 0x7efdc819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 0x7efdc819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42268/0 (socket says 172.21.15.28:42268) 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 -- 172.21.15.28:0/3057169699 learned_addr learned my addr 172.21.15.28:0/3057169699 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdccbf6640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc8100e20 0x7efdc8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdc7fff640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 0x7efdc8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 -- 172.21.15.28:0/3057169699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 msgr2=0x7efdc8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 0x7efdc8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 -- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc8100e20 msgr2=0x7efdc8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc8100e20 0x7efdc8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc819d4b0 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdc7fff640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc8102570 0x7efdc8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.036+0000 7efdcd3f7640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 0x7efdc819cdd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009fd0 tx=0x7efdc000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.037+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0017070 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.037+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc0003e50 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.037+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc819d7a0 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.037+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00106c0 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.037+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc81a5080 con 0x7efdc8102ef0 2024-09-19T09:31:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.038+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdc8106bb0 con 0x7efdc8102ef0 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.040+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efdc0003970 con 0x7efdc8102ef0 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.040+0000 7efdc5ffb640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb80a32d0 0x7efdb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efdc00cfa10 con 0x7efdc8102ef0 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdccbf6640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb80a32d0 0x7efdb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc5ffb640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] conn(0x7efdb80acb20 0x7efdb80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdc001de50 con 0x7efdb80acb20 2024-09-19T09:31:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efdc00cfef0 con 0x7efdc8102ef0 2024-09-19T09:31:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc7fff640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] conn(0x7efdb80acb20 0x7efdb80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdccbf6640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb80a32d0 0x7efdb80a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009230 tx=0x7efdbc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.041+0000 7efdc7fff640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] conn(0x7efdb80acb20 0x7efdb80aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.042+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== osd.10 v2:172.21.15.38:6824/1524737905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdc001de50 con 0x7efdb80acb20 2024-09-19T09:31:20.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.057+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 --> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdc81021b0 con 0x7efdb80acb20 2024-09-19T09:31:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.057+0000 7efdc5ffb640 1 -- 172.21.15.28:0/3057169699 <== osd.10 v2:172.21.15.38:6824/1524737905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efdc81021b0 con 0x7efdb80acb20 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] conn(0x7efdb80acb20 msgr2=0x7efdb80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] conn(0x7efdb80acb20 0x7efdb80aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb80a32d0 msgr2=0x7efdb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb80a32d0 0x7efdb80a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009230 tx=0x7efdbc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 msgr2=0x7efdc819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 --2- 172.21.15.28:0/3057169699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc8102ef0 0x7efdc819cdd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009fd0 tx=0x7efdc000b260 comp rx=0 tx=0).stop 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcd3f7640 1 -- 172.21.15.28:0/3057169699 reap_dead start 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 shutdown_connections 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 >> 172.21.15.28:0/3057169699 conn(0x7efdc80fc7d0 msgr2=0x7efdc806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 shutdown_connections 2024-09-19T09:31:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.058+0000 7efdcee5a640 1 -- 172.21.15.28:0/3057169699 wait complete. 2024-09-19T09:31:20.869 INFO:teuthology.orchestra.run.smithi028.stdout:287762808858 2024-09-19T09:31:20.869 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T09:31:20.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 -- 172.21.15.28:0/2091068703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 msgr2=0x7efdc4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 --2- 172.21.15.28:0/2091068703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc4103120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009900 tx=0x7efdb402a480 comp rx=0 tx=0).stop 2024-09-19T09:31:20.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 -- 172.21.15.28:0/2091068703 shutdown_connections 2024-09-19T09:31:20.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 --2- 172.21.15.28:0/2091068703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 0x7efdc410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 --2- 172.21.15.28:0/2091068703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 0x7efdc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 --2- 172.21.15.28:0/2091068703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc4103120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 -- 172.21.15.28:0/2091068703 >> 172.21.15.28:0/2091068703 conn(0x7efdc40fc7a0 msgr2=0x7efdc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 -- 172.21.15.28:0/2091068703 shutdown_connections 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.630+0000 7efdcadc9640 1 -- 172.21.15.28:0/2091068703 wait complete. 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 Processor -- start 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 -- start start 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4102d40 0x7efdc41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4103660 0x7efdc4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 0x7efdc4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4111f70 con 0x7efdc410c170 2024-09-19T09:31:20.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4111df0 con 0x7efdc4102d40 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.631+0000 7efdcadc9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41120f0 con 0x7efdc4103660 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 0x7efdc4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 0x7efdc4101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42298/0 (socket says 172.21.15.28:42298) 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 -- 172.21.15.28:0/1708214320 learned_addr learned my addr 172.21.15.28:0/1708214320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc3fff640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4103660 0x7efdc4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc8b65640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4102d40 0x7efdc41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 -- 172.21.15.28:0/1708214320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4103660 msgr2=0x7efdc4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4103660 0x7efdc4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 -- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4102d40 msgr2=0x7efdc41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4102d40 0x7efdc41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc41a68f0 con 0x7efdc410c170 2024-09-19T09:31:20.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc9366640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 0x7efdc4101420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00cad0 tx=0x7efdbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00ae20 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc005020 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc41a6be0 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.632+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00f6f0 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.633+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc41a7170 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.633+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdc40628c0 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.635+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efdbc0051d0 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.635+0000 7efdc1ffb640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdac0a3350 0x7efdac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.635+0000 7efdc8b65640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdac0a3350 0x7efdac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.635+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efdbc0cfc10 con 0x7efdc410c170 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc1ffb640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] conn(0x7efdac0acba0 0x7efdac0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:20.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdbc012ec0 con 0x7efdac0acba0 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0d00f0 con 0x7efdc410c170 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc8b65640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdac0a3350 0x7efdac0a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efdb4000c00 tx=0x7efdb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc3fff640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] conn(0x7efdac0acba0 0x7efdac0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.636+0000 7efdc3fff640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] conn(0x7efdac0acba0 0x7efdac0aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.637+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== osd.4 v2:172.21.15.38:6808/465897146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdbc012ec0 con 0x7efdac0acba0 2024-09-19T09:31:20.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.651+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 --> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdc4062cc0 con 0x7efdac0acba0 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.651+0000 7efdc1ffb640 1 -- 172.21.15.28:0/1708214320 <== osd.4 v2:172.21.15.38:6808/465897146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efdc4062cc0 con 0x7efdac0acba0 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] conn(0x7efdac0acba0 msgr2=0x7efdac0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] conn(0x7efdac0acba0 0x7efdac0aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdac0a3350 msgr2=0x7efdac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdac0a3350 0x7efdac0a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efdb4000c00 tx=0x7efdb40058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 msgr2=0x7efdc4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 --2- 172.21.15.28:0/1708214320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc410c170 0x7efdc4101420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00cad0 tx=0x7efdbc00cf90 comp rx=0 tx=0).stop 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdc9366640 1 -- 172.21.15.28:0/1708214320 reap_dead start 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 shutdown_connections 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 >> 172.21.15.28:0/1708214320 conn(0x7efdc40fc7a0 msgr2=0x7efdc40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 shutdown_connections 2024-09-19T09:31:20.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:20.652+0000 7efdcadc9640 1 -- 172.21.15.28:0/1708214320 wait complete. 2024-09-19T09:31:21.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:21 smithi028 ceph-mon[30972]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T09:31:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:21 smithi078 ceph-mon[35619]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T09:31:21.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:21 smithi038 ceph-mon[35271]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T09:31:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:21.800 INFO:teuthology.orchestra.run.smithi028.stdout:137438953528 2024-09-19T09:31:21.800 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 -- 172.21.15.28:0/237087198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 msgr2=0x7f1e3c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 --2- 172.21.15.28:0/237087198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c1067f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009900 tx=0x7f1e2c02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 -- 172.21.15.28:0/237087198 shutdown_connections 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 --2- 172.21.15.28:0/237087198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e3c106f20 0x7f1e3c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 --2- 172.21.15.28:0/237087198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c1067f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 --2- 172.21.15.28:0/237087198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e3c101df0 0x7f1e3c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 -- 172.21.15.28:0/237087198 >> 172.21.15.28:0/237087198 conn(0x7f1e3c0fd8b0 msgr2=0x7f1e3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 -- 172.21.15.28:0/237087198 shutdown_connections 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.105+0000 7f1e4445e640 1 -- 172.21.15.28:0/237087198 wait complete. 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 Processor -- start 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 -- start start 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e3c101df0 0x7f1e3c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e3c106f20 0x7f1e3c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c111f80 con 0x7f1e3c1027a0 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c111e00 con 0x7f1e3c106f20 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e4445e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c112100 con 0x7f1e3c101df0 2024-09-19T09:31:22.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e419f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e419f9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42314/0 (socket says 172.21.15.28:42314) 2024-09-19T09:31:22.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e419f9640 1 -- 172.21.15.28:0/3887457473 learned_addr learned my addr 172.21.15.28:0/3887457473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:22.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e429fb640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e3c106f20 0x7f1e3c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.106+0000 7f1e421fa640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e3c101df0 0x7f1e3c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 -- 172.21.15.28:0/3887457473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e3c101df0 msgr2=0x7f1e3c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e3c101df0 0x7f1e3c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 -- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e3c106f20 msgr2=0x7f1e3c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e3c106f20 0x7f1e3c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c19d3d0 con 0x7f1e3c1027a0 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e419f9640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c198960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c006010 tx=0x7f1e2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c03f030 con 0x7f1e3c1027a0 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2c003ce0 con 0x7f1e3c1027a0 2024-09-19T09:31:22.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c19d660 con 0x7f1e3c1027a0 2024-09-19T09:31:22.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c035070 con 0x7f1e3c1027a0 2024-09-19T09:31:22.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.107+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c1a4fa0 con 0x7f1e3c1027a0 2024-09-19T09:31:22.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.108+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e3c062980 con 0x7f1e3c1027a0 2024-09-19T09:31:22.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.109+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1e2c042030 con 0x7f1e3c1027a0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e337fe640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1e280a32d0 0x7f1e280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e421fa640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1e280a32d0 0x7f1e280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1e2c0f04a0 con 0x7f1e3c1027a0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e337fe640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] conn(0x7f1e280acb20 0x7f1e280aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e2c040e70 con 0x7f1e280acb20 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f1e2c0f0980 con 0x7f1e3c1027a0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e421fa640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1e280a32d0 0x7f1e280a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005fd0 tx=0x7f1e38009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.110+0000 7f1e429fb640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] conn(0x7f1e280acb20 0x7f1e280aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.111+0000 7f1e429fb640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] conn(0x7f1e280acb20 0x7f1e280aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.112+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== osd.1 v2:172.21.15.38:6800/3199007949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e2c040e70 con 0x7f1e280acb20 2024-09-19T09:31:22.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.125+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 --> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e3c062e00 con 0x7f1e280acb20 2024-09-19T09:31:22.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.126+0000 7f1e337fe640 1 -- 172.21.15.28:0/3887457473 <== osd.1 v2:172.21.15.38:6800/3199007949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e3c062e00 con 0x7f1e280acb20 2024-09-19T09:31:22.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.126+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] conn(0x7f1e280acb20 msgr2=0x7f1e280aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.126+0000 7f1e4445e640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] conn(0x7f1e280acb20 0x7f1e280aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.126+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1e280a32d0 msgr2=0x7f1e280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1e280a32d0 0x7f1e280a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005fd0 tx=0x7f1e38009680 comp rx=0 tx=0).stop 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 msgr2=0x7f1e3c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 --2- 172.21.15.28:0/3887457473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e3c1027a0 0x7f1e3c198960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c006010 tx=0x7f1e2c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e429fb640 1 -- 172.21.15.28:0/3887457473 reap_dead start 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 shutdown_connections 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 >> 172.21.15.28:0/3887457473 conn(0x7f1e3c0fd8b0 msgr2=0x7f1e3c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 shutdown_connections 2024-09-19T09:31:22.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.127+0000 7f1e4445e640 1 -- 172.21.15.28:0/3887457473 wait complete. 2024-09-19T09:31:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 -- 172.21.15.28:0/3120954966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c102ef0 msgr2=0x7f624c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 --2- 172.21.15.28:0/3120954966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c102ef0 0x7f624c10da40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6244009ca0 tx=0x7f624402a950 comp rx=0 tx=0).stop 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 -- 172.21.15.28:0/3120954966 shutdown_connections 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 --2- 172.21.15.28:0/3120954966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 0x7f624c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 --2- 172.21.15.28:0/3120954966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c102ef0 0x7f624c10da40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 --2- 172.21.15.28:0/3120954966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c100e20 0x7f624c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 -- 172.21.15.28:0/3120954966 >> 172.21.15.28:0/3120954966 conn(0x7f624c0fc7d0 msgr2=0x7f624c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 -- 172.21.15.28:0/3120954966 shutdown_connections 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.542+0000 7f6250e1c640 1 -- 172.21.15.28:0/3120954966 wait complete. 2024-09-19T09:31:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 Processor -- start 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 -- start start 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 0x7f624c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 0x7f624c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c102ef0 0x7f624c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f624c110f80 con 0x7f624c100e20 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f624c110e00 con 0x7f624c102570 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6250e1c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f624c111100 con 0x7f624c102ef0 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 0x7f624c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 0x7f624c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42332/0 (socket says 172.21.15.28:42332) 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 -- 172.21.15.28:0/181603573 learned_addr learned my addr 172.21.15.28:0/181603573 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624ad9d640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c102ef0 0x7f624c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6249d9b640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 0x7f624c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 -- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c102ef0 msgr2=0x7f624c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c102ef0 0x7f624c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 -- 172.21.15.28:0/181603573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 msgr2=0x7f624c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 0x7f624c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624a59c640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19d440 con 0x7f624c100e20 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f624ad9d640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f624c102ef0 0x7f624c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.543+0000 7f6249d9b640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f624c102570 0x7f624c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f624a59c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 0x7f624c198490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c970 tx=0x7f623c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c007bb0 con 0x7f624c100e20 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623c007d50 con 0x7f624c100e20 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19d730 con 0x7f624c100e20 2024-09-19T09:31:22.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00f450 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.544+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c1a5010 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.545+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f624c062980 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.546+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f623c004fe0 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.547+0000 7f62437fe640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62380a32d0 0x7f62380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.547+0000 7f6249d9b640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62380a32d0 0x7f62380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.547+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f623c0cfb60 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f62437fe640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] conn(0x7f62380acb20 0x7f62380aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62380af5f0 con 0x7f62380acb20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f623c0d0040 con 0x7f624c100e20 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f6249d9b640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62380a32d0 0x7f62380a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6234005d00 tx=0x7f6234009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f624ad9d640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] conn(0x7f62380acb20 0x7f62380aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.548+0000 7f624ad9d640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] conn(0x7f62380acb20 0x7f62380aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:22.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.549+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== osd.11 v2:172.21.15.78:6824/2321429454 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62380af5f0 con 0x7f62380acb20 2024-09-19T09:31:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.567+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 --> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f624c062e00 con 0x7f62380acb20 2024-09-19T09:31:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f62437fe640 1 -- 172.21.15.28:0/181603573 <== osd.11 v2:172.21.15.78:6824/2321429454 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f624c062e00 con 0x7f62380acb20 2024-09-19T09:31:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] conn(0x7f62380acb20 msgr2=0x7f62380aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] conn(0x7f62380acb20 0x7f62380aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62380a32d0 msgr2=0x7f62380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62380a32d0 0x7f62380a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6234005d00 tx=0x7f6234009550 comp rx=0 tx=0).stop 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 msgr2=0x7f624c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f6250e1c640 1 --2- 172.21.15.28:0/181603573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f624c100e20 0x7f624c198490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c970 tx=0x7f623c00ce30 comp rx=0 tx=0).stop 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.568+0000 7f624ad9d640 1 -- 172.21.15.28:0/181603573 reap_dead start 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.569+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 shutdown_connections 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.569+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 >> 172.21.15.28:0/181603573 conn(0x7f624c0fc7d0 msgr2=0x7f624c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.569+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 shutdown_connections 2024-09-19T09:31:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:22.569+0000 7f6250e1c640 1 -- 172.21.15.28:0/181603573 wait complete. 2024-09-19T09:31:23.395 INFO:teuthology.orchestra.run.smithi028.stdout:317827579925 2024-09-19T09:31:23.395 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T09:31:23.552 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:23 smithi028 ceph-mon[30972]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:23.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- 172.21.15.28:0/1253739534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 msgr2=0x7f62d81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1253739534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d81067f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:23.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- 172.21.15.28:0/1253739534 shutdown_connections 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1253739534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d8106f20 0x7f62d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1253739534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d81067f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1253739534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62d8101df0 0x7f62d81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- 172.21.15.28:0/1253739534 >> 172.21.15.28:0/1253739534 conn(0x7f62d80fd8b0 msgr2=0x7f62d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- 172.21.15.28:0/1253739534 shutdown_connections 2024-09-19T09:31:23.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- 172.21.15.28:0/1253739534 wait complete. 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 Processor -- start 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.563+0000 7f62dd65d640 1 -- start start 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d8101df0 0x7f62d81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62d8106f20 0x7f62d819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8111f80 con 0x7f62d81027a0 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8111e00 con 0x7f62d8106f20 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62dd65d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8112100 con 0x7f62d8101df0 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d8198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58930/0 (socket says 172.21.15.28:58930) 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 -- 172.21.15.28:0/1532984697 learned_addr learned my addr 172.21.15.28:0/1532984697 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d6ffd640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d8101df0 0x7f62d81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d77fe640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62d8106f20 0x7f62d819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 -- 172.21.15.28:0/1532984697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d8101df0 msgr2=0x7f62d81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d8101df0 0x7f62d81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 -- 172.21.15.28:0/1532984697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62d8106f20 msgr2=0x7f62d819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62d8106f20 0x7f62d819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d819d490 con 0x7f62d81027a0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.564+0000 7f62d67fc640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d8198a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62cc006010 tx=0x7f62cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.565+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc03f030 con 0x7f62d81027a0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.565+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62cc003ce0 con 0x7f62d81027a0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.565+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc035070 con 0x7f62d81027a0 2024-09-19T09:31:23.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.565+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d819d720 con 0x7f62d81027a0 2024-09-19T09:31:23.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.565+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d819dbb0 con 0x7f62d81027a0 2024-09-19T09:31:23.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.566+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62d8062900 con 0x7f62d81027a0 2024-09-19T09:31:23.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.567+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f62cc042030 con 0x7f62d81027a0 2024-09-19T09:31:23.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62bffff640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62c00a32d0 0x7f62c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f62cc0f0550 con 0x7f62d81027a0 2024-09-19T09:31:23.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62d6ffd640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62c00a32d0 0x7f62c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62bffff640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] conn(0x7f62c00acb20 0x7f62c00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62cc036e90 con 0x7f62c00acb20 2024-09-19T09:31:23.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f62cc0f0900 con 0x7f62d81027a0 2024-09-19T09:31:23.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62d77fe640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] conn(0x7f62c00acb20 0x7f62c00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.568+0000 7f62d6ffd640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62c00a32d0 0x7f62c00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62c80058c0 tx=0x7f62c800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.569+0000 7f62d77fe640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] conn(0x7f62c00acb20 0x7f62c00aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.570+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== osd.6 v2:172.21.15.28:6818/2855006041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62cc036e90 con 0x7f62c00acb20 2024-09-19T09:31:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.591+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 --> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62d8062d80 con 0x7f62c00acb20 2024-09-19T09:31:23.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.591+0000 7f62bffff640 1 -- 172.21.15.28:0/1532984697 <== osd.6 v2:172.21.15.28:6818/2855006041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f62d8062d80 con 0x7f62c00acb20 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.591+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] conn(0x7f62c00acb20 msgr2=0x7f62c00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.591+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] conn(0x7f62c00acb20 0x7f62c00aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62c00a32d0 msgr2=0x7f62c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f62c00a32d0 0x7f62c00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62c80058c0 tx=0x7f62c800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 msgr2=0x7f62d8198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 --2- 172.21.15.28:0/1532984697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62d81027a0 0x7f62d8198a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62cc006010 tx=0x7f62cc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62d77fe640 1 -- 172.21.15.28:0/1532984697 reap_dead start 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 shutdown_connections 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 >> 172.21.15.28:0/1532984697 conn(0x7f62d80fd8b0 msgr2=0x7f62d80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 shutdown_connections 2024-09-19T09:31:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.592+0000 7f62dd65d640 1 -- 172.21.15.28:0/1532984697 wait complete. 2024-09-19T09:31:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:23 smithi078 ceph-mon[35619]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:23.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:23 smithi038 ceph-mon[35271]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 -- 172.21.15.28:0/3572096255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 msgr2=0x7f919c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 --2- 172.21.15.28:0/3572096255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c1021d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f918c009900 tx=0x7f918c02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 -- 172.21.15.28:0/3572096255 shutdown_connections 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 --2- 172.21.15.28:0/3572096255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919c1027a0 0x7f919c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 --2- 172.21.15.28:0/3572096255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 0x7f919c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 --2- 172.21.15.28:0/3572096255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c1021d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 -- 172.21.15.28:0/3572096255 >> 172.21.15.28:0/3572096255 conn(0x7f919c0fd8f0 msgr2=0x7f919c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 -- 172.21.15.28:0/3572096255 shutdown_connections 2024-09-19T09:31:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 -- 172.21.15.28:0/3572096255 wait complete. 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.878+0000 7f91a1b57640 1 Processor -- start 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 -- start start 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919c1027a0 0x7f919c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 0x7f919c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f919c111fa0 con 0x7f919c101df0 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f919c111e20 con 0x7f919c1027a0 2024-09-19T09:31:23.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f91a1b57640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f919c112120 con 0x7f919c106f20 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58944/0 (socket says 172.21.15.28:58944) 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 -- 172.21.15.28:0/867863872 learned_addr learned my addr 172.21.15.28:0/867863872 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919bfff640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 0x7f919c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 -- 172.21.15.28:0/867863872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 msgr2=0x7f919c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 0x7f919c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919affd640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919c1027a0 0x7f919c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 -- 172.21.15.28:0/867863872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919c1027a0 msgr2=0x7f919c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919c1027a0 0x7f919c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919b7fe640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c19d510 con 0x7f919c101df0 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.879+0000 7f919bfff640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106f20 0x7f919c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:23.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f919b7fe640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c198560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f918c005ab0 tx=0x7f918c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c03f030 con 0x7f919c101df0 2024-09-19T09:31:23.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c003ce0 con 0x7f919c101df0 2024-09-19T09:31:23.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c19d7a0 con 0x7f919c101df0 2024-09-19T09:31:23.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c035070 con 0x7f919c101df0 2024-09-19T09:31:23.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.880+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1a50e0 con 0x7f919c101df0 2024-09-19T09:31:23.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.881+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f919c062900 con 0x7f919c101df0 2024-09-19T09:31:23.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.882+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f918c042030 con 0x7f919c101df0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f9198ff9640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91840a3350 0x7f91840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f919affd640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91840a3350 0x7f91840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f918c0f0520 con 0x7f919c101df0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f9198ff9640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] conn(0x7f91840acba0 0x7f91840aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f918c001d30 con 0x7f91840acba0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f918c0f0a00 con 0x7f919c101df0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.883+0000 7f919bfff640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] conn(0x7f91840acba0 0x7f91840aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.884+0000 7f919affd640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91840a3350 0x7f91840a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f919c1088c0 tx=0x7f9190005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.884+0000 7f919bfff640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] conn(0x7f91840acba0 0x7f91840aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:23.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.884+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== osd.9 v2:172.21.15.28:6826/1812921948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f918c001d30 con 0x7f91840acba0 2024-09-19T09:31:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.898+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 --> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f919c062d00 con 0x7f91840acba0 2024-09-19T09:31:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.898+0000 7f9198ff9640 1 -- 172.21.15.28:0/867863872 <== osd.9 v2:172.21.15.28:6826/1812921948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f919c062d00 con 0x7f91840acba0 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] conn(0x7f91840acba0 msgr2=0x7f91840aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] conn(0x7f91840acba0 0x7f91840aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91840a3350 msgr2=0x7f91840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91840a3350 0x7f91840a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f919c1088c0 tx=0x7f9190005c60 comp rx=0 tx=0).stop 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 msgr2=0x7f919c198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 --2- 172.21.15.28:0/867863872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f919c101df0 0x7f919c198560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f918c005ab0 tx=0x7f918c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f919bfff640 1 -- 172.21.15.28:0/867863872 reap_dead start 2024-09-19T09:31:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 shutdown_connections 2024-09-19T09:31:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 >> 172.21.15.28:0/867863872 conn(0x7f919c0fd8f0 msgr2=0x7f919c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 shutdown_connections 2024-09-19T09:31:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:23.899+0000 7f91a1b57640 1 -- 172.21.15.28:0/867863872 wait complete. 2024-09-19T09:31:24.490 INFO:teuthology.orchestra.run.smithi028.stdout:55834574920 2024-09-19T09:31:24.490 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T09:31:25.114 INFO:teuthology.orchestra.run.smithi028.stdout:270582939680 2024-09-19T09:31:25.115 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T09:31:25.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:25 smithi028 ceph-mon[30972]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:25.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:25 smithi078 ceph-mon[35619]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:25.646 INFO:teuthology.orchestra.run.smithi028.stdout:188978561071 2024-09-19T09:31:25.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T09:31:25.656 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:25.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:25 smithi038 ceph-mon[35271]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:26.051 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:26.101 INFO:teuthology.orchestra.run.smithi028.stdout:34359738443 2024-09-19T09:31:26.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T09:31:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 -- 172.21.15.28:0/973464656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 msgr2=0x7fbad410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 --2- 172.21.15.28:0/973464656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad410ec70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009cd0 tx=0x7fbacc02a8d0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 -- 172.21.15.28:0/973464656 shutdown_connections 2024-09-19T09:31:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 --2- 172.21.15.28:0/973464656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad410ec70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 --2- 172.21.15.28:0/973464656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbad4103780 0x7fbad4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 --2- 172.21.15.28:0/973464656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbad4102dd0 0x7fbad41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 -- 172.21.15.28:0/973464656 >> 172.21.15.28:0/973464656 conn(0x7fbad40fc7d0 msgr2=0x7fbad40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 -- 172.21.15.28:0/973464656 shutdown_connections 2024-09-19T09:31:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.746+0000 7fbad88bd640 1 -- 172.21.15.28:0/973464656 wait complete. 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 Processor -- start 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 -- start start 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbad4102dd0 0x7fbad4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbad4103780 0x7fbad4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4112120 con 0x7fbad4108300 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4111fa0 con 0x7fbad4103780 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.747+0000 7fbad88bd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41122a0 con 0x7fbad4102dd0 2024-09-19T09:31:26.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58954/0 (socket says 172.21.15.28:58954) 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 -- 172.21.15.28:0/996180471 learned_addr learned my addr 172.21.15.28:0/996180471 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad1d9b640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbad4103780 0x7fbad4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad259c640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbad4102dd0 0x7fbad4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 -- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbad4102dd0 msgr2=0x7fbad4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbad4102dd0 0x7fbad4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 -- 172.21.15.28:0/996180471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbad4103780 msgr2=0x7fbad4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbad4103780 0x7fbad4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad419d510 con 0x7fbad4108300 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad2d9d640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad419ce30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbacc031040 tx=0x7fbacc031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc031790 con 0x7fbad4108300 2024-09-19T09:31:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad419d7a0 con 0x7fbad4108300 2024-09-19T09:31:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc003e50 con 0x7fbad4108300 2024-09-19T09:31:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0307f0 con 0x7fbad4108300 2024-09-19T09:31:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.748+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a50e0 con 0x7fbad4108300 2024-09-19T09:31:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.749+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad4062900 con 0x7fbad4108300 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.751+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbacc003970 con 0x7fbad4108300 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbac37fe640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbabc0a32d0 0x7fbabc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbad259c640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbabc0a32d0 0x7fbabc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fbacc0f0930 con 0x7fbad4108300 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbac37fe640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] conn(0x7fbabc0acb20 0x7fbabc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbacc04de50 con 0x7fbabc0acb20 2024-09-19T09:31:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0f0ce0 con 0x7fbad4108300 2024-09-19T09:31:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbad259c640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbabc0a32d0 0x7fbabc0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbac400bbe0 tx=0x7fbac400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.752+0000 7fbad1d9b640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] conn(0x7fbabc0acb20 0x7fbabc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.753+0000 7fbad1d9b640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] conn(0x7fbabc0acb20 0x7fbabc0aef40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.754+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== osd.8 v2:172.21.15.78:6816/1616708891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbacc04de50 con 0x7fbabc0acb20 2024-09-19T09:31:26.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.767+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 --> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbad4062d00 con 0x7fbabc0acb20 2024-09-19T09:31:26.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbac37fe640 1 -- 172.21.15.28:0/996180471 <== osd.8 v2:172.21.15.78:6816/1616708891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbad4062d00 con 0x7fbabc0acb20 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] conn(0x7fbabc0acb20 msgr2=0x7fbabc0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] conn(0x7fbabc0acb20 0x7fbabc0aef40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbabc0a32d0 msgr2=0x7fbabc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbabc0a32d0 0x7fbabc0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbac400bbe0 tx=0x7fbac400b390 comp rx=0 tx=0).stop 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 msgr2=0x7fbad419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:26.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 --2- 172.21.15.28:0/996180471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbad4108300 0x7fbad419ce30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbacc031040 tx=0x7fbacc031420 comp rx=0 tx=0).stop 2024-09-19T09:31:26.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad2d9d640 1 -- 172.21.15.28:0/996180471 reap_dead start 2024-09-19T09:31:26.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 shutdown_connections 2024-09-19T09:31:26.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 >> 172.21.15.28:0/996180471 conn(0x7fbad40fc7d0 msgr2=0x7fbad4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:26.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 shutdown_connections 2024-09-19T09:31:26.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.768+0000 7fbad88bd640 1 -- 172.21.15.28:0/996180471 wait complete. 2024-09-19T09:31:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- 172.21.15.28:0/3013561440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a341008d0 msgr2=0x7f6a34100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/3013561440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a341008d0 0x7f6a34100cb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009900 tx=0x7f6a2402a410 comp rx=0 tx=0).stop 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- 172.21.15.28:0/3013561440 shutdown_connections 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/3013561440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a3410e480 0x7f6a34110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/3013561440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a34101280 0x7f6a3410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/3013561440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a341008d0 0x7f6a34100cb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- 172.21.15.28:0/3013561440 >> 172.21.15.28:0/3013561440 conn(0x7f6a340fc7a0 msgr2=0x7f6a340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- 172.21.15.28:0/3013561440 shutdown_connections 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- 172.21.15.28:0/3013561440 wait complete. 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 Processor -- start 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.971+0000 7f6a38dd7640 1 -- start start 2024-09-19T09:31:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a341008d0 0x7f6a3419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a34101280 0x7f6a3419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 0x7f6a3419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114200 con 0x7f6a3410e480 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114080 con 0x7f6a34101280 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a38dd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114380 con 0x7f6a341008d0 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 0x7f6a3419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 0x7f6a3419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58970/0 (socket says 172.21.15.28:58970) 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 -- 172.21.15.28:0/1055807038 learned_addr learned my addr 172.21.15.28:0/1055807038 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a2bfff640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a34101280 0x7f6a3419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a3259c640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a341008d0 0x7f6a3419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 -- 172.21.15.28:0/1055807038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a341008d0 msgr2=0x7f6a3419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a341008d0 0x7f6a3419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 -- 172.21.15.28:0/1055807038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a34101280 msgr2=0x7f6a3419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a34101280 0x7f6a3419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3419f690 con 0x7f6a3410e480 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a2bfff640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a34101280 0x7f6a3419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a3259c640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a341008d0 0x7f6a3419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a32d9d640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 0x7f6a3419efb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009fd0 tx=0x7f6a2c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.972+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004d20 con 0x7f6a3410e480 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.973+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3419f980 con 0x7f6a3410e480 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.973+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c005020 con 0x7f6a3410e480 2024-09-19T09:31:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.973+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0116d0 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.973+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a7260 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.973+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a34062980 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.975+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6a2c0051d0 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.975+0000 7f6a2b7fe640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a200a32d0 0x7f6a200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.975+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6a2c0d00e0 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.975+0000 7f6a3259c640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a200a32d0 0x7f6a200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a2b7fe640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] conn(0x7f6a200acb20 0x7f6a200aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a2c014ec0 con 0x7f6a200acb20 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c0d05c0 con 0x7f6a3410e480 2024-09-19T09:31:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a2bfff640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] conn(0x7f6a200acb20 0x7f6a200aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a3259c640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a200a32d0 0x7f6a200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.976+0000 7f6a2bfff640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] conn(0x7f6a200acb20 0x7f6a200aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.977+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== osd.3 v2:172.21.15.28:6810/3620512995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a2c014ec0 con 0x7f6a200acb20 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.990+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 --> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a34062e00 con 0x7f6a200acb20 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.990+0000 7f6a2b7fe640 1 -- 172.21.15.28:0/1055807038 <== osd.3 v2:172.21.15.28:6810/3620512995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6a34062e00 con 0x7f6a200acb20 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] conn(0x7f6a200acb20 msgr2=0x7f6a200aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] conn(0x7f6a200acb20 0x7f6a200aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a200a32d0 msgr2=0x7f6a200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a200a32d0 0x7f6a200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24032c70 comp rx=0 tx=0).stop 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 msgr2=0x7f6a3419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 --2- 172.21.15.28:0/1055807038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3410e480 0x7f6a3419efb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009fd0 tx=0x7f6a2c00ee60 comp rx=0 tx=0).stop 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a32d9d640 1 -- 172.21.15.28:0/1055807038 reap_dead start 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 shutdown_connections 2024-09-19T09:31:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 >> 172.21.15.28:0/1055807038 conn(0x7f6a340fc7a0 msgr2=0x7f6a340feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 shutdown_connections 2024-09-19T09:31:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:26.991+0000 7f6a38dd7640 1 -- 172.21.15.28:0/1055807038 wait complete. 2024-09-19T09:31:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:27 smithi078 ceph-mon[35619]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:27.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:27 smithi038 ceph-mon[35271]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:27.747 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:27 smithi028 ceph-mon[30972]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T09:31:28.125 INFO:teuthology.orchestra.run.smithi028.stdout:240518168613 2024-09-19T09:31:28.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T09:31:28.579 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:28.580 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:28.580 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:28.581 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:28.619 INFO:teuthology.orchestra.run.smithi028.stdout:103079215167 2024-09-19T09:31:28.619 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T09:31:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 -- 172.21.15.28:0/3655886819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798106f00 msgr2=0x7f379810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 --2- 172.21.15.28:0/3655886819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798106f00 0x7f379810ea80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3790009ca0 tx=0x7f379002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 -- 172.21.15.28:0/3655886819 shutdown_connections 2024-09-19T09:31:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 --2- 172.21.15.28:0/3655886819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798106f00 0x7f379810ea80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 --2- 172.21.15.28:0/3655886819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798102780 0x7f37981067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 --2- 172.21.15.28:0/3655886819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798101dd0 0x7f37981021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 -- 172.21.15.28:0/3655886819 >> 172.21.15.28:0/3655886819 conn(0x7f37980fd8b0 msgr2=0x7f37980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 -- 172.21.15.28:0/3655886819 shutdown_connections 2024-09-19T09:31:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.774+0000 7f379d248640 1 -- 172.21.15.28:0/3655886819 wait complete. 2024-09-19T09:31:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.775+0000 7f379d248640 1 Processor -- start 2024-09-19T09:31:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.775+0000 7f379d248640 1 -- start start 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 0x7f3798198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798102780 0x7f37981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798106f00 0x7f379819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111f80 con 0x7f3798101dd0 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111e00 con 0x7f3798102780 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379d248640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3798112100 con 0x7f3798106f00 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 0x7f3798198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 0x7f3798198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58998/0 (socket says 172.21.15.28:58998) 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 -- 172.21.15.28:0/3772374655 learned_addr learned my addr 172.21.15.28:0/3772374655 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379659c640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798102780 0x7f37981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379759e640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798106f00 0x7f379819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 -- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798106f00 msgr2=0x7f379819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798106f00 0x7f379819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 -- 172.21.15.28:0/3772374655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798102780 msgr2=0x7f37981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798102780 0x7f37981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f3796d9d640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379819d410 con 0x7f3798101dd0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379759e640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3798106f00 0x7f379819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.776+0000 7f379659c640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3798102780 0x7f37981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f3796d9d640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 0x7f3798198460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f378800d950 tx=0x7f378800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788017070 con 0x7f3798101dd0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3788009c30 con 0x7f3798101dd0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379819d700 con 0x7f3798101dd0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800f6b0 con 0x7f3798101dd0 2024-09-19T09:31:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.777+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a4fe0 con 0x7f3798101dd0 2024-09-19T09:31:28.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.778+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3798062980 con 0x7f3798101dd0 2024-09-19T09:31:28.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.781+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3788010460 con 0x7f3798101dd0 2024-09-19T09:31:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f377ffff640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f37800a3350 0x7f37800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f37880cfa60 con 0x7f3798101dd0 2024-09-19T09:31:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f379659c640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f37800a3350 0x7f37800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f377ffff640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] conn(0x7f37800acba0 0x7f37800aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3788005600 con 0x7f37800acba0 2024-09-19T09:31:28.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.782+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f37880d42a0 con 0x7f3798101dd0 2024-09-19T09:31:28.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.783+0000 7f379759e640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] conn(0x7f37800acba0 0x7f37800aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:28.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.783+0000 7f379659c640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f37800a3350 0x7f37800a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37980fdcd0 tx=0x7f378c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:28.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.784+0000 7f379759e640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] conn(0x7f37800acba0 0x7f37800aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:28.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.785+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== osd.2 v2:172.21.15.78:6800/720593762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3788005600 con 0x7f37800acba0 2024-09-19T09:31:28.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.800+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 --> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3798062e00 con 0x7f37800acba0 2024-09-19T09:31:28.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.800+0000 7f377ffff640 1 -- 172.21.15.28:0/3772374655 <== osd.2 v2:172.21.15.78:6800/720593762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3798062e00 con 0x7f37800acba0 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] conn(0x7f37800acba0 msgr2=0x7f37800aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] conn(0x7f37800acba0 0x7f37800aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f37800a3350 msgr2=0x7f37800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f37800a3350 0x7f37800a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37980fdcd0 tx=0x7f378c008040 comp rx=0 tx=0).stop 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 msgr2=0x7f3798198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 --2- 172.21.15.28:0/3772374655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3798101dd0 0x7f3798198460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f378800d950 tx=0x7f378800de10 comp rx=0 tx=0).stop 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379759e640 1 -- 172.21.15.28:0/3772374655 reap_dead start 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 shutdown_connections 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 >> 172.21.15.28:0/3772374655 conn(0x7f37980fd8b0 msgr2=0x7f37980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 shutdown_connections 2024-09-19T09:31:28.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:28.801+0000 7f379d248640 1 -- 172.21.15.28:0/3772374655 wait complete. 2024-09-19T09:31:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:29.308 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:29 smithi028 ceph-mon[30972]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:31:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 -- 172.21.15.28:0/3106033166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8100e20 msgr2=0x7f04e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 --2- 172.21.15.28:0/3106033166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8100e20 0x7f04e8101200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009900 tx=0x7f04d802a480 comp rx=0 tx=0).stop 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 -- 172.21.15.28:0/3106033166 shutdown_connections 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 --2- 172.21.15.28:0/3106033166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 --2- 172.21.15.28:0/3106033166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8102570 0x7f04e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 --2- 172.21.15.28:0/3106033166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8100e20 0x7f04e8101200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 -- 172.21.15.28:0/3106033166 >> 172.21.15.28:0/3106033166 conn(0x7f04e80fc7d0 msgr2=0x7f04e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.391+0000 7f04ed023640 1 -- 172.21.15.28:0/3106033166 shutdown_connections 2024-09-19T09:31:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.392+0000 7f04ed023640 1 -- 172.21.15.28:0/3106033166 wait complete. 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.392+0000 7f04ed023640 1 Processor -- start 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 -- start start 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8100e20 0x7f04e8116560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 0x7f04e81105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810f040 con 0x7f04e8102570 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810eec0 con 0x7f04e8102ef0 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04ed023640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810f1c0 con 0x7f04e8100e20 2024-09-19T09:31:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04e6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8100e20 0x7f04e8116560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04e759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04e759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:50448/0 (socket says 172.21.15.28:50448) 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04e759e640 1 -- 172.21.15.28:0/2210457888 learned_addr learned my addr 172.21.15.28:0/2210457888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.393+0000 7f04e659c640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 0x7f04e81105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 -- 172.21.15.28:0/2210457888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8100e20 msgr2=0x7f04e8116560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8100e20 0x7f04e8116560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 -- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 msgr2=0x7f04e81105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 0x7f04e81105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 -- 172.21.15.28:0/2210457888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e8111220 con 0x7f04e8102ef0 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e659c640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 0x7f04e81105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04e759e640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04e000eaa0 tx=0x7f04e000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e000cdb0 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0005020 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e81b39d0 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00116b0 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.394+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81b3e30 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.395+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e81021b0 con 0x7f04e8102ef0 2024-09-19T09:31:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.397+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f04e0005560 con 0x7f04e8102ef0 2024-09-19T09:31:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.397+0000 7f04cffff640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 0x7f04d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.397+0000 7f04e6d9d640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 0x7f04d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.397+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f04e00cffb0 con 0x7f04e8102ef0 2024-09-19T09:31:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.398+0000 7f04e6d9d640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 0x7f04d00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009230 tx=0x7f04d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:29.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.401+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04e0016030 con 0x7f04e8102ef0 2024-09-19T09:31:29.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.577+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f04e81113f0 con 0x7f04e8102ef0 2024-09-19T09:31:29.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.578+0000 7f04cffff640 1 -- 172.21.15.28:0/2210457888 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f04e009cc20 con 0x7f04e8102ef0 2024-09-19T09:31:29.580 INFO:teuthology.orchestra.run.smithi028.stdout:287762808859 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 msgr2=0x7f04d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 0x7f04d00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009230 tx=0x7f04d80058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 msgr2=0x7f04e8110b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04e000eaa0 tx=0x7f04e000ef60 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 shutdown_connections 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04d00a32d0 0x7f04d00a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04e8102ef0 0x7f04e8110b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04e8102570 0x7f04e81105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 --2- 172.21.15.28:0/2210457888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e8100e20 0x7f04e8116560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 >> 172.21.15.28:0/2210457888 conn(0x7f04e80fc7d0 msgr2=0x7f04e80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 shutdown_connections 2024-09-19T09:31:29.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:29.580+0000 7f04ed023640 1 -- 172.21.15.28:0/2210457888 wait complete. 2024-09-19T09:31:29.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:29 smithi078 ceph-mon[35619]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:31:29.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:29 smithi038 ceph-mon[35271]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:31:29.849 INFO:teuthology.orchestra.run.smithi028.stdout:77309411396 2024-09-19T09:31:29.849 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T09:31:30.071 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:30.151 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:30.409 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:30 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2210457888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:31:30.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 -- 172.21.15.28:0/3706742696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed5010e4d0 msgr2=0x7fed501108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:30.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 --2- 172.21.15.28:0/3706742696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed5010e4d0 0x7fed501108e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed48009ca0 tx=0x7fed4802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 -- 172.21.15.28:0/3706742696 shutdown_connections 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 --2- 172.21.15.28:0/3706742696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed50101360 0x7fed5010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 --2- 172.21.15.28:0/3706742696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed5010e4d0 0x7fed501108e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 --2- 172.21.15.28:0/3706742696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed501009b0 0x7fed50100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 -- 172.21.15.28:0/3706742696 >> 172.21.15.28:0/3706742696 conn(0x7fed500fc7d0 msgr2=0x7fed500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 -- 172.21.15.28:0/3706742696 shutdown_connections 2024-09-19T09:31:30.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.569+0000 7fed58211640 1 -- 172.21.15.28:0/3706742696 wait complete. 2024-09-19T09:31:30.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.570+0000 7fed58211640 1 Processor -- start 2024-09-19T09:31:30.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.570+0000 7fed58211640 1 -- start start 2024-09-19T09:31:30.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.570+0000 7fed58211640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:30.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.570+0000 7fed58211640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 0x7fed5019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.570+0000 7fed58211640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed58211640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed50114220 con 0x7fed501009b0 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed58211640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed501140a0 con 0x7fed5010e4d0 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed58211640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed501143a0 con 0x7fed50101360 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed557ac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 0x7fed5019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed567ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59042/0 (socket says 172.21.15.28:59042) 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed567ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:50470/0 (socket says 172.21.15.28:50470) 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 -- 172.21.15.28:0/3947826862 learned_addr learned my addr 172.21.15.28:0/3947826862 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:30.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 -- 172.21.15.28:0/3947826862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 msgr2=0x7fed5019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 0x7fed5019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 -- 172.21.15.28:0/3947826862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 msgr2=0x7fed5019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 -- 172.21.15.28:0/3947826862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5019f6f0 con 0x7fed501009b0 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed557ac640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 0x7fed5019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed567ae640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed55fad640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00e9f0 tx=0x7fed4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00cd80 con 0x7fed501009b0 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c005500 con 0x7fed501009b0 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c010690 con 0x7fed501009b0 2024-09-19T09:31:30.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.571+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5019f9e0 con 0x7fed501009b0 2024-09-19T09:31:30.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.572+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed501a72c0 con 0x7fed501009b0 2024-09-19T09:31:30.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.573+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50100d90 con 0x7fed501009b0 2024-09-19T09:31:30.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.574+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fed4c0046c0 con 0x7fed501009b0 2024-09-19T09:31:30.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.575+0000 7fed46ffd640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 0x7fed3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:30.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.575+0000 7fed557ac640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 0x7fed3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:30.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.575+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fed4c0cff50 con 0x7fed501009b0 2024-09-19T09:31:30.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.576+0000 7fed557ac640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 0x7fed3c0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed50102790 tx=0x7fed40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:30.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.579+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed4c054210 con 0x7fed501009b0 2024-09-19T09:31:30.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:30 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2210457888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:31:30.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2210457888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:31:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.763+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fed5019baa0 con 0x7fed501009b0 2024-09-19T09:31:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.763+0000 7fed46ffd640 1 -- 172.21.15.28:0/3947826862 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed4c09cbc0 con 0x7fed501009b0 2024-09-19T09:31:30.765 INFO:teuthology.orchestra.run.smithi028.stdout:210453397546 2024-09-19T09:31:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 msgr2=0x7fed3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 0x7fed3c0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed50102790 tx=0x7fed40005f90 comp rx=0 tx=0).stop 2024-09-19T09:31:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 msgr2=0x7fed5019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00e9f0 tx=0x7fed4c00edf0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 shutdown_connections 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fed3c0a3350 0x7fed3c0a5810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed5010e4d0 0x7fed5019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed50101360 0x7fed5019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 --2- 172.21.15.28:0/3947826862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed501009b0 0x7fed5019a740 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 >> 172.21.15.28:0/3947826862 conn(0x7fed500fc7d0 msgr2=0x7fed500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 shutdown_connections 2024-09-19T09:31:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:30.765+0000 7fed58211640 1 -- 172.21.15.28:0/3947826862 wait complete. 2024-09-19T09:31:30.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808858 got 287762808859 for osd.10 2024-09-19T09:31:30.982 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:31.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 -- 172.21.15.28:0/3979409585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c103660 msgr2=0x7ff78c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 --2- 172.21.15.28:0/3979409585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c103660 0x7ff78c10bb50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009900 tx=0x7ff77c02a480 comp rx=0 tx=0).stop 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 -- 172.21.15.28:0/3979409585 shutdown_connections 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 --2- 172.21.15.28:0/3979409585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 0x7ff78c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 --2- 172.21.15.28:0/3979409585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c103660 0x7ff78c10bb50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 --2- 172.21.15.28:0/3979409585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c102d40 0x7ff78c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.095+0000 7ff79336b640 1 -- 172.21.15.28:0/3979409585 >> 172.21.15.28:0/3979409585 conn(0x7ff78c0fc7a0 msgr2=0x7ff78c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- 172.21.15.28:0/3979409585 shutdown_connections 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- 172.21.15.28:0/3979409585 wait complete. 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 Processor -- start 2024-09-19T09:31:31.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- start start 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c103660 0x7ff78c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 0x7ff78c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c111fb0 con 0x7ff78c102d40 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c111e30 con 0x7ff78c10c170 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.096+0000 7ff79336b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c112130 con 0x7ff78c103660 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59060/0 (socket says 172.21.15.28:59060) 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 -- 172.21.15.28:0/624289753 learned_addr learned my addr 172.21.15.28:0/624289753 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791908640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 0x7ff78c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff790906640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c103660 0x7ff78c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:31.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 -- 172.21.15.28:0/624289753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c103660 msgr2=0x7ff78c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c103660 0x7ff78c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 -- 172.21.15.28:0/624289753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 msgr2=0x7ff78c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 0x7ff78c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 -- 172.21.15.28:0/624289753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c1a6a80 con 0x7ff78c102d40 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff791107640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b4a0 tx=0x7ff78800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788014070 con 0x7ff78c102d40 2024-09-19T09:31:31.428 INFO:teuthology.orchestra.run.smithi028.stdout:158913790004 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff788010af0 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c1a6d70 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788017d10 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.097+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1a7250 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.098+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c103120 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.099+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff788017070 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.100+0000 7ff7827fc640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 0x7ff7780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.100+0000 7ff790906640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 0x7ff7780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.100+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff7880ceb40 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.101+0000 7ff790906640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 0x7ff7780a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0096c0 tx=0x7ff77c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.105+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff78809b9c0 con 0x7ff78c102d40 2024-09-19T09:31:31.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.283+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff78c1071e0 con 0x7ff78c102d40 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.284+0000 7ff7827fc640 1 -- 172.21.15.28:0/624289753 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff78809b2c0 con 0x7ff78c102d40 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 msgr2=0x7ff7780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 0x7ff7780a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0096c0 tx=0x7ff77c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 msgr2=0x7ff78c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b4a0 tx=0x7ff78800b960 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 shutdown_connections 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7780a3350 0x7ff7780a5810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff78c10c170 0x7ff78c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c103660 0x7ff78c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 --2- 172.21.15.28:0/624289753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78c102d40 0x7ff78c1009a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 >> 172.21.15.28:0/624289753 conn(0x7ff78c0fc7a0 msgr2=0x7ff78c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:31.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 shutdown_connections 2024-09-19T09:31:31.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:31.287+0000 7ff79336b640 1 -- 172.21.15.28:0/624289753 wait complete. 2024-09-19T09:31:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:31 smithi078 ceph-mon[35619]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:31:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:31 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3947826862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:31:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:31 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/624289753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:31:31.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:31 smithi038 ceph-mon[35271]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:31:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:31 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3947826862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:31:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:31 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/624289753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:31:31.710 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:31 smithi028 ceph-mon[30972]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:31:31.710 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:31 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3947826862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:31:31.710 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:31 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/624289753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 -- 172.21.15.28:0/3801547711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0102790 msgr2=0x7fc5f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3801547711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0102790 0x7fc5f01067e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 -- 172.21.15.28:0/3801547711 shutdown_connections 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3801547711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3801547711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0102790 0x7fc5f01067e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3801547711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0101de0 0x7fc5f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 -- 172.21.15.28:0/3801547711 >> 172.21.15.28:0/3801547711 conn(0x7fc5f00fd8c0 msgr2=0x7fc5f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:32.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 -- 172.21.15.28:0/3801547711 shutdown_connections 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.119+0000 7fc5f7174640 1 -- 172.21.15.28:0/3801547711 wait complete. 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 Processor -- start 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 -- start start 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 0x7fc5f006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111f90 con 0x7fc5f0101de0 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111e10 con 0x7fc5f0106f10 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f7174640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0112110 con 0x7fc5f0102790 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59080/0 (socket says 172.21.15.28:59080) 2024-09-19T09:31:32.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 -- 172.21.15.28:0/3598895334 learned_addr learned my addr 172.21.15.28:0/3598895334 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5effff640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 0x7fc5f006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f5711640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 -- 172.21.15.28:0/3598895334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 msgr2=0x7fc5f006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 0x7fc5f006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 -- 172.21.15.28:0/3598895334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 msgr2=0x7fc5f006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 -- 172.21.15.28:0/3598895334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f006d580 con 0x7fc5f0101de0 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5effff640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 0x7fc5f006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f5711640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.120+0000 7fc5f4f10640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b4a0 tx=0x7fc5e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.121+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4014070 con 0x7fc5f0101de0 2024-09-19T09:31:32.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.121+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f006d870 con 0x7fc5f0101de0 2024-09-19T09:31:32.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.121+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e4010af0 con 0x7fc5f0101de0 2024-09-19T09:31:32.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.121+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4017d10 con 0x7fc5f0101de0 2024-09-19T09:31:32.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.121+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f0071830 con 0x7fc5f0101de0 2024-09-19T09:31:32.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.122+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0072310 con 0x7fc5f0101de0 2024-09-19T09:31:32.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.123+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc5e4017070 con 0x7fc5f0101de0 2024-09-19T09:31:32.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.124+0000 7fc5edffb640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 0x7fc5e00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.124+0000 7fc5effff640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 0x7fc5e00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.124+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc5e40ceb80 con 0x7fc5f0101de0 2024-09-19T09:31:32.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.124+0000 7fc5effff640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 0x7fc5e00a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5f006e2b0 tx=0x7fc5dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:32.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.128+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc5e409ba00 con 0x7fc5f0101de0 2024-09-19T09:31:32.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.307+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc5f0109b60 con 0x7fc5f0101de0 2024-09-19T09:31:32.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.307+0000 7fc5edffb640 1 -- 172.21.15.28:0/3598895334 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc5e409b300 con 0x7fc5f0101de0 2024-09-19T09:31:32.309 INFO:teuthology.orchestra.run.smithi028.stdout:317827579927 2024-09-19T09:31:32.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 msgr2=0x7fc5e00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 0x7fc5e00a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5f006e2b0 tx=0x7fc5dc005a20 comp rx=0 tx=0).stop 2024-09-19T09:31:32.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 msgr2=0x7fc5f00739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b4a0 tx=0x7fc5e400b960 comp rx=0 tx=0).stop 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 shutdown_connections 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc5e00a3350 0x7fc5e00a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f0106f10 0x7fc5f006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f0102790 0x7fc5f006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 --2- 172.21.15.28:0/3598895334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5f0101de0 0x7fc5f00739d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 >> 172.21.15.28:0/3598895334 conn(0x7fc5f00fd8c0 msgr2=0x7fc5f00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 shutdown_connections 2024-09-19T09:31:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.310+0000 7fc5f7174640 1 -- 172.21.15.28:0/3598895334 wait complete. 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 -- 172.21.15.28:0/3285633284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090102790 msgr2=0x7f20901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3285633284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090102790 0x7f20901067e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2084009900 tx=0x7f208402a410 comp rx=0 tx=0).stop 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 -- 172.21.15.28:0/3285633284 shutdown_connections 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3285633284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f209010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3285633284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090102790 0x7f20901067e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3285633284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090101de0 0x7f20901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 -- 172.21.15.28:0/3285633284 >> 172.21.15.28:0/3285633284 conn(0x7f20900fd8c0 msgr2=0x7f20900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 -- 172.21.15.28:0/3285633284 shutdown_connections 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 -- 172.21.15.28:0/3285633284 wait complete. 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.317+0000 7f2095ef0640 1 Processor -- start 2024-09-19T09:31:32.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 -- start start 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111fc0 con 0x7f2090101de0 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111e40 con 0x7f2090106f10 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f2095ef0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2090112140 con 0x7f2090102790 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59106/0 (socket says 172.21.15.28:59106) 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f208effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f208f7fe640 1 -- 172.21.15.28:0/3994075998 learned_addr learned my addr 172.21.15.28:0/3994075998 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.318+0000 7f208ffff640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 -- 172.21.15.28:0/3994075998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090102790 msgr2=0x7f2090074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 -- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 msgr2=0x7f20900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 -- 172.21.15.28:0/3994075998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a68e0 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208f7fe640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208ffff640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f2090074f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2088009fd0 tx=0x7f208800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800ce20 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2088005020 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a6bd0 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088010720 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.319+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a7090 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.320+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20901021c0 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.321+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2088005540 con 0x7f2090106f10 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.322+0000 7f208cff9640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.322+0000 7f208f7fe640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:32.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.322+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f20880cff60 con 0x7f2090106f10 2024-09-19T09:31:32.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.323+0000 7f208f7fe640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 0x7f20780a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20800104e0 tx=0x7f208000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:32.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.327+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f208809ccd0 con 0x7f2090106f10 2024-09-19T09:31:32.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.511+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f209010a150 con 0x7f2090106f10 2024-09-19T09:31:32.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.512+0000 7f208cff9640 1 -- 172.21.15.28:0/3994075998 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f208809c5d0 con 0x7f2090106f10 2024-09-19T09:31:32.514 INFO:teuthology.orchestra.run.smithi028.stdout:55834574922 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 msgr2=0x7f20780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 0x7f20780a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20800104e0 tx=0x7f208000b410 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 msgr2=0x7f2090074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f2090074f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2088009fd0 tx=0x7f208800ef10 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 shutdown_connections 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 --2- 172.21.15.28:0/3994075998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 >> 172.21.15.28:0/3994075998 conn(0x7f20900fd8c0 msgr2=0x7f20900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:32.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 shutdown_connections 2024-09-19T09:31:32.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:32.514+0000 7f2095ef0640 1 -- 172.21.15.28:0/3994075998 wait complete. 2024-09-19T09:31:33.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2024-09-19T09:31:33.092 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:33.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:33 smithi028 ceph-mon[30972]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:33.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:33 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3598895334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:31:33.324 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:33 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3994075998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:31:33.329 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:33 smithi078 ceph-mon[35619]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:33 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3598895334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:31:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:33 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3994075998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:31:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:33 smithi038 ceph-mon[35271]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:33 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3598895334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:31:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:33 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3994075998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:31:33.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 -- 172.21.15.28:0/3410813955 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 msgr2=0x7fd0e81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:33.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 --2- 172.21.15.28:0/3410813955 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81021d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009900 tx=0x7fd0e402a410 comp rx=0 tx=0).stop 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 -- 172.21.15.28:0/3410813955 shutdown_connections 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 --2- 172.21.15.28:0/3410813955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e81027a0 0x7fd0e81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 --2- 172.21.15.28:0/3410813955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e8106f20 0x7fd0e810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 --2- 172.21.15.28:0/3410813955 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81021d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 -- 172.21.15.28:0/3410813955 >> 172.21.15.28:0/3410813955 conn(0x7fd0e80fd8f0 msgr2=0x7fd0e80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:33.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 -- 172.21.15.28:0/3410813955 shutdown_connections 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 -- 172.21.15.28:0/3410813955 wait complete. 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.852+0000 7fd0f0096640 1 Processor -- start 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 -- start start 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e81027a0 0x7fd0e8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:33.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 0x7fd0e819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8111f90 con 0x7fd0e8101df0 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8111e10 con 0x7fd0e8106f20 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0f0096640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8112110 con 0x7fd0e81027a0 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40024/0 (socket says 172.21.15.28:40024) 2024-09-19T09:31:33.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 -- 172.21.15.28:0/222118575 learned_addr learned my addr 172.21.15.28:0/222118575 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:33.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ed631640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e81027a0 0x7fd0e8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:33.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ee633640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 0x7fd0e819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:33.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 -- 172.21.15.28:0/222118575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e81027a0 msgr2=0x7fd0e8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:33.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e81027a0 0x7fd0e8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:33.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 -- 172.21.15.28:0/222118575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 msgr2=0x7fd0e819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:33.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 0x7fd0e819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:33.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ede32640 1 -- 172.21.15.28:0/222118575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e819d3a0 con 0x7fd0e8101df0 2024-09-19T09:31:33.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.853+0000 7fd0ee633640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 0x7fd0e819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:33.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0ede32640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4004770 tx=0x7fd0e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:33.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e403f070 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e819d630 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e4031ca0 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4035030 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.854+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a4f70 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.855+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e81032d0 con 0x7fd0e8101df0 2024-09-19T09:31:33.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.856+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd0e4042070 con 0x7fd0e8101df0 2024-09-19T09:31:33.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.857+0000 7fd0deffd640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 0x7fd0d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:33.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.857+0000 7fd0ed631640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 0x7fd0d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:33.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.857+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd0e40f01d0 con 0x7fd0e8101df0 2024-09-19T09:31:33.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.858+0000 7fd0ed631640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 0x7fd0d40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0e81088d0 tx=0x7fd0d800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:33.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.862+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0e40bce40 con 0x7fd0e8101df0 2024-09-19T09:31:34.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.057+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd0e810a100 con 0x7fd0e8101df0 2024-09-19T09:31:34.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.058+0000 7fd0deffd640 1 -- 172.21.15.28:0/222118575 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0e40bc740 con 0x7fd0e8101df0 2024-09-19T09:31:34.060 INFO:teuthology.orchestra.run.smithi028.stdout:270582939681 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 msgr2=0x7fd0d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 0x7fd0d40a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0e81088d0 tx=0x7fd0d800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 msgr2=0x7fd0e81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4004770 tx=0x7fd0e4003c10 comp rx=0 tx=0).stop 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 shutdown_connections 2024-09-19T09:31:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd0d40a32d0 0x7fd0d40a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0e8106f20 0x7fd0e819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0e81027a0 0x7fd0e8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 --2- 172.21.15.28:0/222118575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e8101df0 0x7fd0e81983f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 >> 172.21.15.28:0/222118575 conn(0x7fd0e80fd8f0 msgr2=0x7fd0e80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 shutdown_connections 2024-09-19T09:31:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:34.060+0000 7fd0f0096640 1 -- 172.21.15.28:0/222118575 wait complete. 2024-09-19T09:31:34.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579925 got 317827579927 for osd.11 2024-09-19T09:31:34.260 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:34 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1651544089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:31:34.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:34 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/222118575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:31:34.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:34 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1651544089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:31:34.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:34 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/222118575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:31:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:34 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1651544089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:31:34.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:34 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/222118575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:31:35.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 -- 172.21.15.28:0/723163516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 msgr2=0x7f0da810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/723163516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da810ea90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009ca0 tx=0x7f0da002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 -- 172.21.15.28:0/723163516 shutdown_connections 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/723163516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da810ea90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/723163516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8102790 0x7f0da81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/723163516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8101de0 0x7f0da81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 -- 172.21.15.28:0/723163516 >> 172.21.15.28:0/723163516 conn(0x7f0da80fd8c0 msgr2=0x7f0da80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 -- 172.21.15.28:0/723163516 shutdown_connections 2024-09-19T09:31:35.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 -- 172.21.15.28:0/723163516 wait complete. 2024-09-19T09:31:35.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.010+0000 7f0dad1e8640 1 Processor -- start 2024-09-19T09:31:35.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 -- start start 2024-09-19T09:31:35.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8101de0 0x7f0da8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8102790 0x7f0da8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8111f90 con 0x7f0da8106f10 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8111e10 con 0x7f0da8102790 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0dad1e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8112110 con 0x7f0da8101de0 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40036/0 (socket says 172.21.15.28:40036) 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 -- 172.21.15.28:0/901916535 learned_addr learned my addr 172.21.15.28:0/901916535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da659c640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8102790 0x7f0da8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da6d9d640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8101de0 0x7f0da8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 -- 172.21.15.28:0/901916535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8101de0 msgr2=0x7f0da8198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8101de0 0x7f0da8198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 -- 172.21.15.28:0/901916535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8102790 msgr2=0x7f0da8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8102790 0x7f0da8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.011+0000 7f0da759e640 1 -- 172.21.15.28:0/901916535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da819d500 con 0x7f0da8106f10 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0da759e640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0da0031a60 tx=0x7f0da0031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0003b70 con 0x7f0da8106f10 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da0003d10 con 0x7f0da8106f10 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da819d790 con 0x7f0da8106f10 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0004770 con 0x7f0da8106f10 2024-09-19T09:31:35.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.012+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a50d0 con 0x7f0da8106f10 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.013+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da81021c0 con 0x7f0da8106f10 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.014+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0da002f790 con 0x7f0da8106f10 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.015+0000 7f0d8ffff640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 0x7f0d900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.015+0000 7f0da6d9d640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 0x7f0d900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.015+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0da00f0c50 con 0x7f0da8106f10 2024-09-19T09:31:35.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.015+0000 7f0da6d9d640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 0x7f0d900a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d980058c0 tx=0x7f0d9800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:35.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.018+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0da00bdad0 con 0x7f0da8106f10 2024-09-19T09:31:35.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.749+0000 7f1542a84640 1 -- 172.21.15.28:0/3891560788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c10c170 msgr2=0x7f153c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.749+0000 7f1542a84640 1 --2- 172.21.15.28:0/3891560788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c10c170 0x7f153c10e610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1534009ca0 tx=0x7f153402a950 comp rx=0 tx=0).stop 2024-09-19T09:31:35.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 -- 172.21.15.28:0/3891560788 shutdown_connections 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 --2- 172.21.15.28:0/3891560788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c10c170 0x7f153c10e610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 --2- 172.21.15.28:0/3891560788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 0x7f153c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 --2- 172.21.15.28:0/3891560788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f153c102d40 0x7f153c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 -- 172.21.15.28:0/3891560788 >> 172.21.15.28:0/3891560788 conn(0x7f153c0fc7a0 msgr2=0x7f153c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 -- 172.21.15.28:0/3891560788 shutdown_connections 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 -- 172.21.15.28:0/3891560788 wait complete. 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 Processor -- start 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.750+0000 7f1542a84640 1 -- start start 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 0x7f153c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f153c10c170 0x7f153c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f153c111fb0 con 0x7f153c102d40 2024-09-19T09:31:35.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f153c111e30 con 0x7f153c10c170 2024-09-19T09:31:35.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1542a84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f153c112130 con 0x7f153c103660 2024-09-19T09:31:35.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.090 INFO:teuthology.orchestra.run.smithi028.stdout:34359738446 2024-09-19T09:31:35.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40008/0 (socket says 172.21.15.28:40008) 2024-09-19T09:31:35.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 -- 172.21.15.28:0/1651544089 learned_addr learned my addr 172.21.15.28:0/1651544089 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 -- 172.21.15.28:0/1651544089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 msgr2=0x7f153c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f153bfff640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 0x7f153c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 0x7f153c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 -- 172.21.15.28:0/1651544089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f153c10c170 msgr2=0x7f153c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f153c10c170 0x7f153c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 -- 172.21.15.28:0/1651544089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c19d3e0 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1540820640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b4a0 tx=0x7f152c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.751+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c014070 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.752+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152c010af0 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.752+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c19d6d0 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.752+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c017d10 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.752+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1a4fb0 con 0x7f153c102d40 2024-09-19T09:31:35.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.753+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c103120 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.754+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f152c017070 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.755+0000 7f1539ffb640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 0x7f15280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.755+0000 7f153bfff640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 0x7f15280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.755+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f152c0cec00 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.755+0000 7f153bfff640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 0x7f15280a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f1524005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.759+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f152c0d4020 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.945+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f153c199790 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.945+0000 7f1539ffb640 1 -- 172.21.15.28:0/1651544089 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f152c09ba80 con 0x7f153c102d40 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 msgr2=0x7f15280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 0x7f15280a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f1524005d00 comp rx=0 tx=0).stop 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 msgr2=0x7f153c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b4a0 tx=0x7f152c00b960 comp rx=0 tx=0).stop 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 shutdown_connections 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f15280a32d0 0x7f15280a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f153c10c170 0x7f153c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c103660 0x7f153c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 --2- 172.21.15.28:0/1651544089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f153c102d40 0x7f153c198430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 >> 172.21.15.28:0/1651544089 conn(0x7f153c0fc7a0 msgr2=0x7f153c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.948+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 shutdown_connections 2024-09-19T09:31:35.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:33.949+0000 7f1542a84640 1 -- 172.21.15.28:0/1651544089 wait complete. 2024-09-19T09:31:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.196+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0da81998b0 con 0x7f0da8106f10 2024-09-19T09:31:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.197+0000 7f0d8ffff640 1 -- 172.21.15.28:0/901916535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0da81998b0 con 0x7f0da8106f10 2024-09-19T09:31:35.199 INFO:teuthology.orchestra.run.smithi028.stdout:137438953531 2024-09-19T09:31:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 msgr2=0x7f0d900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 0x7f0d900a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d980058c0 tx=0x7f0d9800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 msgr2=0x7f0da819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0da0031a60 tx=0x7f0da0031a90 comp rx=0 tx=0).stop 2024-09-19T09:31:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 shutdown_connections 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0d900a32d0 0x7f0d900a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da8102790 0x7f0da8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da8106f10 0x7f0da819ce20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 --2- 172.21.15.28:0/901916535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0da8101de0 0x7f0da8198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 >> 172.21.15.28:0/901916535 conn(0x7f0da80fd8c0 msgr2=0x7f0da80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 shutdown_connections 2024-09-19T09:31:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:35.199+0000 7f0dad1e8640 1 -- 172.21.15.28:0/901916535 wait complete. 2024-09-19T09:31:35.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:35 smithi028 ceph-mon[30972]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:31:35.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:35 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/901916535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:31:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:35 smithi078 ceph-mon[35619]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:31:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:35 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/901916535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:31:35.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2024-09-19T09:31:35.674 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:35.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:35 smithi038 ceph-mon[35271]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:31:35.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:35 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/901916535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:31:36.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397544 got 210453397546 for osd.7 2024-09-19T09:31:36.041 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:36.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790002 got 158913790004 for osd.5 2024-09-19T09:31:36.492 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:36.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939681 for osd.9 2024-09-19T09:31:36.745 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:37.413 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:37 smithi028 ceph-mon[30972]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 -- 172.21.15.28:0/1929929839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 msgr2=0x7f7c0c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1929929839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c10ea90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009ca0 tx=0x7f7c0402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 -- 172.21.15.28:0/1929929839 shutdown_connections 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1929929839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 0x7f7c0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1929929839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c10ea90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1929929839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 -- 172.21.15.28:0/1929929839 >> 172.21.15.28:0/1929929839 conn(0x7f7c0c0fd8c0 msgr2=0x7f7c0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:37.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 -- 172.21.15.28:0/1929929839 shutdown_connections 2024-09-19T09:31:37.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.496+0000 7f7c1288f640 1 -- 172.21.15.28:0/1929929839 wait complete. 2024-09-19T09:31:37.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 Processor -- start 2024-09-19T09:31:37.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 -- start start 2024-09-19T09:31:37.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:37.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 0x7f7c0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:37.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111f70 con 0x7f7c0c106f10 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111df0 con 0x7f7c0c102790 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c1288f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1120f0 con 0x7f7c0c101de0 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c10e2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c10e2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40054/0 (socket says 172.21.15.28:40054) 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c10e2c640 1 -- 172.21.15.28:0/1073194342 learned_addr learned my addr 172.21.15.28:0/1073194342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c0b7fe640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 0x7f7c0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.497+0000 7f7c0bfff640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 -- 172.21.15.28:0/1073194342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 msgr2=0x7f7c0c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 -- 172.21.15.28:0/1073194342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 msgr2=0x7f7c0c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:37.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 0x7f7c0c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 -- 172.21.15.28:0/1073194342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1a6890 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c0bfff640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c10e2c640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c04031a20 tx=0x7f7c04031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04003c30 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1a6b20 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04003dd0 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0402fcf0 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.498+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a70b0 con 0x7f7c0c106f10 2024-09-19T09:31:37.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.499+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c1021c0 con 0x7f7c0c106f10 2024-09-19T09:31:37.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.501+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7c04004b40 con 0x7f7c0c106f10 2024-09-19T09:31:37.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.502+0000 7f7c097fa640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 0x7f7bf40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:37.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.502+0000 7f7c0bfff640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 0x7f7bf40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:37.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.502+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7c040f0da0 con 0x7f7c0c106f10 2024-09-19T09:31:37.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.502+0000 7f7c0bfff640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 0x7f7bf40a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0058c0 tx=0x7f7bfc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:37.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.505+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c040bdbb0 con 0x7f7c0c106f10 2024-09-19T09:31:37.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:37 smithi078 ceph-mon[35619]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T09:31:37.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:37 smithi038 ceph-mon[35271]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T09:31:37.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.683+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7c0c02d7c0 con 0x7f7c0c106f10 2024-09-19T09:31:37.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.684+0000 7f7c097fa640 1 -- 172.21.15.28:0/1073194342 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c0402fe90 con 0x7f7c0c106f10 2024-09-19T09:31:37.685 INFO:teuthology.orchestra.run.smithi028.stdout:188978561073 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 msgr2=0x7f7bf40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 0x7f7bf40a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0058c0 tx=0x7f7bfc005850 comp rx=0 tx=0).stop 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 msgr2=0x7f7c0c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c04031a20 tx=0x7f7c04031a50 comp rx=0 tx=0).stop 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 shutdown_connections 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7bf40a32d0 0x7f7bf40a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c0c102790 0x7f7c0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c106f10 0x7f7c0c074f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 --2- 172.21.15.28:0/1073194342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c0c101de0 0x7f7c0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 >> 172.21.15.28:0/1073194342 conn(0x7f7c0c0fd8c0 msgr2=0x7f7c0c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 shutdown_connections 2024-09-19T09:31:37.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:37.686+0000 7f7c1288f640 1 -- 172.21.15.28:0/1073194342 wait complete. 2024-09-19T09:31:37.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953528 got 137438953531 for osd.4 2024-09-19T09:31:37.839 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:38.403 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:38 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1073194342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:31:38.403 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:38 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:31:38.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561073 for osd.6 2024-09-19T09:31:38.460 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:38.575 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:38 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1073194342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:31:38.575 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:38 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:31:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:38 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1073194342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:31:38.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:38 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:31:39.595 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:39 smithi028 ceph-mon[30972]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:31:39.595 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.595 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.595 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.595 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:39 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:39 smithi078 ceph-mon[35619]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:31:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:39 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:39 smithi038 ceph-mon[35271]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:31:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:39 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:39.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.752+0000 7f65d0206640 1 -- 172.21.15.28:0/2145754605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 msgr2=0x7f65c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.752+0000 7f65d0206640 1 --2- 172.21.15.28:0/2145754605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c810ea90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009ca0 tx=0x7f65c002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 -- 172.21.15.28:0/2145754605 shutdown_connections 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 --2- 172.21.15.28:0/2145754605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c810ea90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 --2- 172.21.15.28:0/2145754605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102790 0x7f65c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 --2- 172.21.15.28:0/2145754605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8101de0 0x7f65c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 -- 172.21.15.28:0/2145754605 >> 172.21.15.28:0/2145754605 conn(0x7f65c80fd8c0 msgr2=0x7f65c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 -- 172.21.15.28:0/2145754605 shutdown_connections 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 -- 172.21.15.28:0/2145754605 wait complete. 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 Processor -- start 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 -- start start 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.753+0000 7f65d0206640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 0x7f65c8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f65c8111f70 con 0x7f65c8106f10 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65c8111df0 con 0x7f65c8102790 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81120f0 con 0x7f65c8101de0 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65ce7a3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:55032/0 (socket says 172.21.15.28:55032) 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 -- 172.21.15.28:0/2424228423 learned_addr learned my addr 172.21.15.28:0/2424228423 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cd7a1640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 0x7f65c8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 -- 172.21.15.28:0/2424228423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 msgr2=0x7f65c8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 0x7f65c8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 -- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 msgr2=0x7f65c819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 -- 172.21.15.28:0/2424228423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c819d380 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65ce7a3640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cd7a1640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 0x7f65c8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65cdfa2640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00d950 tx=0x7f65bc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc017070 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc009c30 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c819d670 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00f6b0 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.754+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a4f50 con 0x7f65c8101de0 2024-09-19T09:31:39.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.755+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c81021c0 con 0x7f65c8101de0 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.757+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f65bc0109d0 con 0x7f65c8101de0 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.757+0000 7f65baffd640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 0x7f65b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.757+0000 7f65cd7a1640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 0x7f65b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.757+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f65bc0cf610 con 0x7f65c8101de0 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.758+0000 7f65cd7a1640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 0x7f65b40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65c81088e0 tx=0x7f65c40094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:39.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.760+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f65bc009de0 con 0x7f65c8101de0 2024-09-19T09:31:39.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.940+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f65c802d7c0 con 0x7f65c8101de0 2024-09-19T09:31:39.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.941+0000 7f65baffd640 1 -- 172.21.15.28:0/2424228423 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65bc09c280 con 0x7f65c8101de0 2024-09-19T09:31:39.943 INFO:teuthology.orchestra.run.smithi028.stdout:77309411398 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 msgr2=0x7f65b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 0x7f65b40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65c81088e0 tx=0x7f65c40094b0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 msgr2=0x7f65c81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00d950 tx=0x7f65bc00de10 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 shutdown_connections 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f65b40a32d0 0x7f65b40a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65c8102790 0x7f65c8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65c8106f10 0x7f65c819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 --2- 172.21.15.28:0/2424228423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8101de0 0x7f65c81983d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 >> 172.21.15.28:0/2424228423 conn(0x7f65c80fd8c0 msgr2=0x7f65c80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 shutdown_connections 2024-09-19T09:31:39.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.943+0000 7f65d0206640 1 -- 172.21.15.28:0/2424228423 wait complete. 2024-09-19T09:31:39.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 -- 172.21.15.28:0/1073958129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 msgr2=0x7f56181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 --2- 172.21.15.28:0/1073958129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181067e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f560c009900 tx=0x7f560c02a410 comp rx=0 tx=0).stop 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 -- 172.21.15.28:0/1073958129 shutdown_connections 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 --2- 172.21.15.28:0/1073958129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618106f10 0x7f561810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 --2- 172.21.15.28:0/1073958129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181067e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 --2- 172.21.15.28:0/1073958129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618101de0 0x7f56181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 -- 172.21.15.28:0/1073958129 >> 172.21.15.28:0/1073958129 conn(0x7f56180fd8c0 msgr2=0x7f56180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 -- 172.21.15.28:0/1073958129 shutdown_connections 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.973+0000 7f561e519640 1 -- 172.21.15.28:0/1073958129 wait complete. 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 Processor -- start 2024-09-19T09:31:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 -- start start 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618101de0 0x7f56181984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618106f10 0x7f561819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5618111f90 con 0x7f5618102790 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5618111e10 con 0x7f5618101de0 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f561e519640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5618112110 con 0x7f5618106f10 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.974+0000 7f56177fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40082/0 (socket says 172.21.15.28:40082) 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 -- 172.21.15.28:0/1092093248 learned_addr learned my addr 172.21.15.28:0/1092093248 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f561cab6640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618106f10 0x7f561819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f5617fff640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618101de0 0x7f56181984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 -- 172.21.15.28:0/1092093248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618106f10 msgr2=0x7f561819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618106f10 0x7f561819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 -- 172.21.15.28:0/1092093248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618101de0 msgr2=0x7f56181984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618101de0 0x7f56181984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 -- 172.21.15.28:0/1092093248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819d460 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56177fe640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f560c005ab0 tx=0x7f560c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c03f030 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819d6f0 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560c003ce0 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c035070 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.975+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a5030 con 0x7f5618102790 2024-09-19T09:31:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.977+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56181021c0 con 0x7f5618102790 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.978+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f560c042030 con 0x7f5618102790 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.979+0000 7f56157fa640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 0x7f56000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.979+0000 7f5617fff640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 0x7f56000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.979+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f560c0f04e0 con 0x7f5618102790 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.979+0000 7f5617fff640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 0x7f56000a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f560800abe0 tx=0x7f560800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:39.982+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f560c0bd150 con 0x7f5618102790 2024-09-19T09:31:40.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.174+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f561810a0f0 con 0x7f5618102790 2024-09-19T09:31:40.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.175+0000 7f56157fa640 1 -- 172.21.15.28:0/1092093248 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f560c0bca50 con 0x7f5618102790 2024-09-19T09:31:40.177 INFO:teuthology.orchestra.run.smithi028.stdout:103079215169 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 msgr2=0x7f56000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 0x7f56000a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f560800abe0 tx=0x7f560800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 msgr2=0x7f56181989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f560c005ab0 tx=0x7f560c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 shutdown_connections 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f56000a32d0 0x7f56000a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5618106f10 0x7f561819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5618102790 0x7f56181989f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 --2- 172.21.15.28:0/1092093248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5618101de0 0x7f56181984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:40.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 >> 172.21.15.28:0/1092093248 conn(0x7f56180fd8c0 msgr2=0x7f56180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:40.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 shutdown_connections 2024-09-19T09:31:40.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:40.177+0000 7f561e519640 1 -- 172.21.15.28:0/1092093248 wait complete. 2024-09-19T09:31:40.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:40 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2424228423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:31:40.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:40 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1092093248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:31:40.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:40 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2424228423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:31:40.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:40 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1092093248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:31:40.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:40 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2424228423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:31:40.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:40 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1092093248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:31:40.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411398 for osd.2 2024-09-19T09:31:40.786 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:41.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215169 for osd.3 2024-09-19T09:31:41.264 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 -- 172.21.15.28:0/3782218791 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630100e20 msgr2=0x7f9630101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 --2- 172.21.15.28:0/3782218791 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630100e20 0x7f9630101200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9620009900 tx=0x7f962002a480 comp rx=0 tx=0).stop 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 -- 172.21.15.28:0/3782218791 shutdown_connections 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 --2- 172.21.15.28:0/3782218791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630102ef0 0x7f963010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 --2- 172.21.15.28:0/3782218791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 0x7f96301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 --2- 172.21.15.28:0/3782218791 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630100e20 0x7f9630101200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 -- 172.21.15.28:0/3782218791 >> 172.21.15.28:0/3782218791 conn(0x7f96300fc7d0 msgr2=0x7f96300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 -- 172.21.15.28:0/3782218791 shutdown_connections 2024-09-19T09:31:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.469+0000 7f9637228640 1 -- 172.21.15.28:0/3782218791 wait complete. 2024-09-19T09:31:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 Processor -- start 2024-09-19T09:31:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 -- start start 2024-09-19T09:31:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630100e20 0x7f9630199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 0x7f9630199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9630110f80 con 0x7f9630102ef0 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9630110e00 con 0x7f9630100e20 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9637228640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9630111100 con 0x7f9630102570 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f96357c5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f9634fc4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630100e20 0x7f9630199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f96357c5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40102/0 (socket says 172.21.15.28:40102) 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f96357c5640 1 -- 172.21.15.28:0/4024396161 learned_addr learned my addr 172.21.15.28:0/4024396161 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.470+0000 7f962ffff640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 0x7f9630199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 -- 172.21.15.28:0/4024396161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 msgr2=0x7f9630199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 0x7f9630199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 -- 172.21.15.28:0/4024396161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630100e20 msgr2=0x7f9630199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630100e20 0x7f9630199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 -- 172.21.15.28:0/4024396161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96301a49d0 con 0x7f9630102ef0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f96357c5640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f962800e970 tx=0x7f962800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962800ccc0 con 0x7f9630102ef0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962800ce60 con 0x7f9630102ef0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96301a4ba0 con 0x7f9630102ef0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628010490 con 0x7f9630102ef0 2024-09-19T09:31:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.471+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96301a5060 con 0x7f9630102ef0 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.473+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9630101200 con 0x7f9630102ef0 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.474+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9628005020 con 0x7f9630102ef0 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.474+0000 7f962dffb640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 0x7f96180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.474+0000 7f9634fc4640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 0x7f96180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.474+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f96280cfd10 con 0x7f9630102ef0 2024-09-19T09:31:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.475+0000 7f9634fc4640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 0x7f96180a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9620009230 tx=0x7f96200058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:41.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.479+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f962809ca80 con 0x7f9630102ef0 2024-09-19T09:31:41.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:41 smithi078 ceph-mon[35619]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:31:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:41 smithi038 ceph-mon[35271]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:31:41.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.674+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9630109090 con 0x7f9630102ef0 2024-09-19T09:31:41.676 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:41 smithi028 ceph-mon[30972]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:31:41.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.674+0000 7f962dffb640 1 -- 172.21.15.28:0/4024396161 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f962809c380 con 0x7f9630102ef0 2024-09-19T09:31:41.677 INFO:teuthology.orchestra.run.smithi028.stdout:240518168616 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 msgr2=0x7f96180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 0x7f96180a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9620009230 tx=0x7f96200058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 msgr2=0x7f96301a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f962800e970 tx=0x7f962800ed70 comp rx=0 tx=0).stop 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 shutdown_connections 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96180a32d0 0x7f96180a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9630102ef0 0x7f96301a4490 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630102570 0x7f9630199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 --2- 172.21.15.28:0/4024396161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9630100e20 0x7f9630199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:41.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 >> 172.21.15.28:0/4024396161 conn(0x7f96300fc7d0 msgr2=0x7f96300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:41.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 shutdown_connections 2024-09-19T09:31:41.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:41.677+0000 7f9637228640 1 -- 172.21.15.28:0/4024396161 wait complete. 2024-09-19T09:31:42.309 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168616 for osd.8 2024-09-19T09:31:42.309 DEBUG:teuthology.parallel:result is None 2024-09-19T09:31:42.309 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:31:42.310 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:31:42.582 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:42.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:42 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4024396161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:31:42.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:42 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4024396161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:31:42.704 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:42 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4024396161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:31:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/1789162263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4106a80 msgr2=0x7f5cc4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/1789162263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4106a80 0x7f5cc4106e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009930 tx=0x7f5cb402a420 comp rx=0 tx=0).stop 2024-09-19T09:31:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/1789162263 shutdown_connections 2024-09-19T09:31:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/1789162263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4100a80 0x7f5cc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/1789162263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4101420 0x7f5cc410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/1789162263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4106a80 0x7f5cc4106e60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/1789162263 >> 172.21.15.28:0/1789162263 conn(0x7f5cc40fc7a0 msgr2=0x7f5cc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/1789162263 shutdown_connections 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.044+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/1789162263 wait complete. 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 Processor -- start 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 -- start start 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4100a80 0x7f5cc41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4106a80 0x7f5cc419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4110e80 con 0x7f5cc4101420 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4110d00 con 0x7f5cc4106a80 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc9a7c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4111000 con 0x7f5cc4100a80 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40116/0 (socket says 172.21.15.28:40116) 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 -- 172.21.15.28:0/3087446811 learned_addr learned my addr 172.21.15.28:0/3087446811 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc37fe640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4100a80 0x7f5cc41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc3fff640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4106a80 0x7f5cc419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 -- 172.21.15.28:0/3087446811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4100a80 msgr2=0x7f5cc41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4100a80 0x7f5cc41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 -- 172.21.15.28:0/3087446811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4106a80 msgr2=0x7f5cc419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4106a80 0x7f5cc419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.045+0000 7f5cc2ffd640 1 -- 172.21.15.28:0/3087446811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc419d450 con 0x7f5cc4101420 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc2ffd640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800d950 tx=0x7f5cb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8017070 con 0x7f5cc4101420 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc419d740 con 0x7f5cc4101420 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8009c30 con 0x7f5cc4101420 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb800f6b0 con 0x7f5cc4101420 2024-09-19T09:31:43.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.046+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a5020 con 0x7f5cc4101420 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.047+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc4100ee0 con 0x7f5cc4101420 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.048+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5cb80109d0 con 0x7f5cc4101420 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.049+0000 7f5cc0ff9640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 0x7f5cac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.049+0000 7f5cc37fe640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 0x7f5cac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.049+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5cb80cf560 con 0x7f5cc4101420 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.049+0000 7f5cc37fe640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 0x7f5cac0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.053+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cb8009de0 con 0x7f5cc4101420 2024-09-19T09:31:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.227+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cc406afc0 con 0x7f5cac0a32d0 2024-09-19T09:31:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.233+0000 7f5cc0ff9640 1 -- 172.21.15.28:0/3087446811 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258747 (secure 0 0 0) 0x7f5cc406afc0 con 0x7f5cac0a32d0 2024-09-19T09:31:43.239 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:43.240 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 msgr2=0x7f5cac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 0x7f5cac0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb40058d0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 msgr2=0x7f5cc41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800d950 tx=0x7f5cb800de10 comp rx=0 tx=0).stop 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 shutdown_connections 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cac0a32d0 0x7f5cac0a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc4106a80 0x7f5cc419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cc4101420 0x7f5cc41989e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 --2- 172.21.15.28:0/3087446811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc4100a80 0x7f5cc41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:43.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 >> 172.21.15.28:0/3087446811 conn(0x7f5cc40fc7a0 msgr2=0x7f5cc406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:43.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 shutdown_connections 2024-09-19T09:31:43.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:43.236+0000 7f5cc9a7c640 1 -- 172.21.15.28:0/3087446811 wait complete. 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:31:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:31:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:31:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:31:43.939 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-19T09:31:41.409467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657967,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":616,"num_read_kb":428,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365756,"kb_used_data":12348,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576068,"statfs":{"total":1151940427776,"available":1151565893632,"internally_reserved":0,"allocated":12644352,"data_stored":6941697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003035"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565004+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:30:43.565004+0000","last_peered":"2024-09-19T09:30:43.565004+0000","last_clean":"2024-09-19T09:30:43.565004+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:30:43.565004+0000","last_undegraded":"2024-09-19T09:30:43.565004+0000","last_fullsized":"2024-09-19T09:30:43.565004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563865+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:30:43.563865+0000","last_peered":"2024-09-19T09:30:43.563865+0000","last_clean":"2024-09-19T09:30:43.563865+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:30:43.563865+0000","last_undegraded":"2024-09-19T09:30:43.563865+0000","last_fullsized":"2024-09-19T09:30:43.563865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564249+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:30:43.564249+0000","last_peered":"2024-09-19T09:30:43.564249+0000","last_clean":"2024-09-19T09:30:43.564249+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:30:43.564249+0000","last_undegraded":"2024-09-19T09:30:43.564249+0000","last_fullsized":"2024-09-19T09:30:43.564249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561198+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:30:43.561198+0000","last_peered":"2024-09-19T09:30:43.561198+0000","last_clean":"2024-09-19T09:30:43.561198+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:30:43.561198+0000","last_undegraded":"2024-09-19T09:30:43.561198+0000","last_fullsized":"2024-09-19T09:30:43.561198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561252+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:30:43.561252+0000","last_peered":"2024-09-19T09:30:43.561252+0000","last_clean":"2024-09-19T09:30:43.561252+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:30:43.561252+0000","last_undegraded":"2024-09-19T09:30:43.561252+0000","last_fullsized":"2024-09-19T09:30:43.561252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.547978+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:30:43.547978+0000","last_peered":"2024-09-19T09:30:43.547978+0000","last_clean":"2024-09-19T09:30:43.547978+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:30:43.547978+0000","last_undegraded":"2024-09-19T09:30:43.547978+0000","last_fullsized":"2024-09-19T09:30:43.547978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562172+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:30:43.562172+0000","last_peered":"2024-09-19T09:30:43.562172+0000","last_clean":"2024-09-19T09:30:43.562172+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:30:43.562172+0000","last_undegraded":"2024-09-19T09:30:43.562172+0000","last_fullsized":"2024-09-19T09:30:43.562172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542518+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:30:43.542518+0000","last_peered":"2024-09-19T09:30:43.542518+0000","last_clean":"2024-09-19T09:30:43.542518+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:30:43.542518+0000","last_undegraded":"2024-09-19T09:30:43.542518+0000","last_fullsized":"2024-09-19T09:30:43.542518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562779+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:30:43.562779+0000","last_peered":"2024-09-19T09:30:43.562779+0000","last_clean":"2024-09-19T09:30:43.562779+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:30:43.562779+0000","last_undegraded":"2024-09-19T09:30:43.562779+0000","last_fullsized":"2024-09-19T09:30:43.562779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548341+0000","last_change":"2024-09-19T09:30:42.564174+0000","last_active":"2024-09-19T09:30:43.548341+0000","last_peered":"2024-09-19T09:30:43.548341+0000","last_clean":"2024-09-19T09:30:43.548341+0000","last_became_active":"2024-09-19T09:30:42.563790+0000","last_became_peered":"2024-09-19T09:30:42.563790+0000","last_unstale":"2024-09-19T09:30:43.548341+0000","last_undegraded":"2024-09-19T09:30:43.548341+0000","last_fullsized":"2024-09-19T09:30:43.548341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:22.463896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564526+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:30:43.564526+0000","last_peered":"2024-09-19T09:30:43.564526+0000","last_clean":"2024-09-19T09:30:43.564526+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:30:43.564526+0000","last_undegraded":"2024-09-19T09:30:43.564526+0000","last_fullsized":"2024-09-19T09:30:43.564526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561947+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:30:43.561947+0000","last_peered":"2024-09-19T09:30:43.561947+0000","last_clean":"2024-09-19T09:30:43.561947+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:30:43.561947+0000","last_undegraded":"2024-09-19T09:30:43.561947+0000","last_fullsized":"2024-09-19T09:30:43.561947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561657+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:30:43.561657+0000","last_peered":"2024-09-19T09:30:43.561657+0000","last_clean":"2024-09-19T09:30:43.561657+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:30:43.561657+0000","last_undegraded":"2024-09-19T09:30:43.561657+0000","last_fullsized":"2024-09-19T09:30:43.561657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562189+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:30:43.562189+0000","last_peered":"2024-09-19T09:30:43.562189+0000","last_clean":"2024-09-19T09:30:43.562189+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:30:43.562189+0000","last_undegraded":"2024-09-19T09:30:43.562189+0000","last_fullsized":"2024-09-19T09:30:43.562189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562374+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:30:43.562374+0000","last_peered":"2024-09-19T09:30:43.562374+0000","last_clean":"2024-09-19T09:30:43.562374+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:30:43.562374+0000","last_undegraded":"2024-09-19T09:30:43.562374+0000","last_fullsized":"2024-09-19T09:30:43.562374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560910+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:30:43.560910+0000","last_peered":"2024-09-19T09:30:43.560910+0000","last_clean":"2024-09-19T09:30:43.560910+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:30:43.560910+0000","last_undegraded":"2024-09-19T09:30:43.560910+0000","last_fullsized":"2024-09-19T09:30:43.560910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565188+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:30:43.565188+0000","last_peered":"2024-09-19T09:30:43.565188+0000","last_clean":"2024-09-19T09:30:43.565188+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:30:43.565188+0000","last_undegraded":"2024-09-19T09:30:43.565188+0000","last_fullsized":"2024-09-19T09:30:43.565188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560190+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:30:43.560190+0000","last_peered":"2024-09-19T09:30:43.560190+0000","last_clean":"2024-09-19T09:30:43.560190+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:30:43.560190+0000","last_undegraded":"2024-09-19T09:30:43.560190+0000","last_fullsized":"2024-09-19T09:30:43.560190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563025+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:30:43.563025+0000","last_peered":"2024-09-19T09:30:43.563025+0000","last_clean":"2024-09-19T09:30:43.563025+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:30:43.563025+0000","last_undegraded":"2024-09-19T09:30:43.563025+0000","last_fullsized":"2024-09-19T09:30:43.563025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564597+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:30:43.564597+0000","last_peered":"2024-09-19T09:30:43.564597+0000","last_clean":"2024-09-19T09:30:43.564597+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:30:43.564597+0000","last_undegraded":"2024-09-19T09:30:43.564597+0000","last_fullsized":"2024-09-19T09:30:43.564597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561822+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:30:43.561822+0000","last_peered":"2024-09-19T09:30:43.561822+0000","last_clean":"2024-09-19T09:30:43.561822+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:30:43.561822+0000","last_undegraded":"2024-09-19T09:30:43.561822+0000","last_fullsized":"2024-09-19T09:30:43.561822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561430+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:30:43.561430+0000","last_peered":"2024-09-19T09:30:43.561430+0000","last_clean":"2024-09-19T09:30:43.561430+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:30:43.561430+0000","last_undegraded":"2024-09-19T09:30:43.561430+0000","last_fullsized":"2024-09-19T09:30:43.561430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542541+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:30:43.542541+0000","last_peered":"2024-09-19T09:30:43.542541+0000","last_clean":"2024-09-19T09:30:43.542541+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:30:43.542541+0000","last_undegraded":"2024-09-19T09:30:43.542541+0000","last_fullsized":"2024-09-19T09:30:43.542541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544194+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:30:43.544194+0000","last_peered":"2024-09-19T09:30:43.544194+0000","last_clean":"2024-09-19T09:30:43.544194+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:30:43.544194+0000","last_undegraded":"2024-09-19T09:30:43.544194+0000","last_fullsized":"2024-09-19T09:30:43.544194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542969+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:30:43.542969+0000","last_peered":"2024-09-19T09:30:43.542969+0000","last_clean":"2024-09-19T09:30:43.542969+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:30:43.542969+0000","last_undegraded":"2024-09-19T09:30:43.542969+0000","last_fullsized":"2024-09-19T09:30:43.542969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565943+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:30:43.565943+0000","last_peered":"2024-09-19T09:30:43.565943+0000","last_clean":"2024-09-19T09:30:43.565943+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:30:43.565943+0000","last_undegraded":"2024-09-19T09:30:43.565943+0000","last_fullsized":"2024-09-19T09:30:43.565943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564893+0000","last_change":"2024-09-19T09:30:42.559055+0000","last_active":"2024-09-19T09:30:43.564893+0000","last_peered":"2024-09-19T09:30:43.564893+0000","last_clean":"2024-09-19T09:30:43.564893+0000","last_became_active":"2024-09-19T09:30:42.558537+0000","last_became_peered":"2024-09-19T09:30:42.558537+0000","last_unstale":"2024-09-19T09:30:43.564893+0000","last_undegraded":"2024-09-19T09:30:43.564893+0000","last_fullsized":"2024-09-19T09:30:43.564893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.777026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562079+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:30:43.562079+0000","last_peered":"2024-09-19T09:30:43.562079+0000","last_clean":"2024-09-19T09:30:43.562079+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:30:43.562079+0000","last_undegraded":"2024-09-19T09:30:43.562079+0000","last_fullsized":"2024-09-19T09:30:43.562079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564360+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:30:43.564360+0000","last_peered":"2024-09-19T09:30:43.564360+0000","last_clean":"2024-09-19T09:30:43.564360+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:30:43.564360+0000","last_undegraded":"2024-09-19T09:30:43.564360+0000","last_fullsized":"2024-09-19T09:30:43.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562024+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:30:43.562024+0000","last_peered":"2024-09-19T09:30:43.562024+0000","last_clean":"2024-09-19T09:30:43.562024+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:30:43.562024+0000","last_undegraded":"2024-09-19T09:30:43.562024+0000","last_fullsized":"2024-09-19T09:30:43.562024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563172+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:30:43.563172+0000","last_peered":"2024-09-19T09:30:43.563172+0000","last_clean":"2024-09-19T09:30:43.563172+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:30:43.563172+0000","last_undegraded":"2024-09-19T09:30:43.563172+0000","last_fullsized":"2024-09-19T09:30:43.563172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563982+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:30:43.563982+0000","last_peered":"2024-09-19T09:30:43.563982+0000","last_clean":"2024-09-19T09:30:43.563982+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:30:43.563982+0000","last_undegraded":"2024-09-19T09:30:43.563982+0000","last_fullsized":"2024-09-19T09:30:43.563982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560987+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:30:43.560987+0000","last_peered":"2024-09-19T09:30:43.560987+0000","last_clean":"2024-09-19T09:30:43.560987+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:30:43.560987+0000","last_undegraded":"2024-09-19T09:30:43.560987+0000","last_fullsized":"2024-09-19T09:30:43.560987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:39.255347+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:31:39.255347+0000","last_peered":"2024-09-19T09:31:39.255347+0000","last_clean":"2024-09-19T09:31:39.255347+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:31:39.255347+0000","last_undegraded":"2024-09-19T09:31:39.255347+0000","last_fullsized":"2024-09-19T09:31:39.255347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544051+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:30:43.544051+0000","last_peered":"2024-09-19T09:30:43.544051+0000","last_clean":"2024-09-19T09:30:43.544051+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:30:43.544051+0000","last_undegraded":"2024-09-19T09:30:43.544051+0000","last_fullsized":"2024-09-19T09:30:43.544051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184065+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:30:52.184065+0000","last_peered":"2024-09-19T09:30:52.184065+0000","last_clean":"2024-09-19T09:30:52.184065+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:30:52.184065+0000","last_undegraded":"2024-09-19T09:30:52.184065+0000","last_fullsized":"2024-09-19T09:30:52.184065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.163943+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:30:51.163943+0000","last_peered":"2024-09-19T09:30:51.163943+0000","last_clean":"2024-09-19T09:30:51.163943+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:30:51.163943+0000","last_undegraded":"2024-09-19T09:30:51.163943+0000","last_fullsized":"2024-09-19T09:30:51.163943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542591+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:30:43.542591+0000","last_peered":"2024-09-19T09:30:43.542591+0000","last_clean":"2024-09-19T09:30:43.542591+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:30:43.542591+0000","last_undegraded":"2024-09-19T09:30:43.542591+0000","last_fullsized":"2024-09-19T09:30:43.542591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187883+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:30:53.187883+0000","last_peered":"2024-09-19T09:30:53.187883+0000","last_clean":"2024-09-19T09:30:53.187883+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:30:53.187883+0000","last_undegraded":"2024-09-19T09:30:53.187883+0000","last_fullsized":"2024-09-19T09:30:53.187883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.559954+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:30:43.559954+0000","last_peered":"2024-09-19T09:30:43.559954+0000","last_clean":"2024-09-19T09:30:43.559954+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:30:43.559954+0000","last_undegraded":"2024-09-19T09:30:43.559954+0000","last_fullsized":"2024-09-19T09:30:43.559954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":112,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.253519+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:31:38.253519+0000","last_peered":"2024-09-19T09:31:38.253519+0000","last_clean":"2024-09-19T09:31:38.253519+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:31:38.253519+0000","last_undegraded":"2024-09-19T09:31:38.253519+0000","last_fullsized":"2024-09-19T09:31:38.253519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562997+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:30:43.562997+0000","last_peered":"2024-09-19T09:30:43.562997+0000","last_clean":"2024-09-19T09:30:43.562997+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:30:43.562997+0000","last_undegraded":"2024-09-19T09:30:43.562997+0000","last_fullsized":"2024-09-19T09:30:43.562997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187517+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:30:53.187517+0000","last_peered":"2024-09-19T09:30:53.187517+0000","last_clean":"2024-09-19T09:30:53.187517+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:30:53.187517+0000","last_undegraded":"2024-09-19T09:30:53.187517+0000","last_fullsized":"2024-09-19T09:30:53.187517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564521+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:30:43.564521+0000","last_peered":"2024-09-19T09:30:43.564521+0000","last_clean":"2024-09-19T09:30:43.564521+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:30:43.564521+0000","last_undegraded":"2024-09-19T09:30:43.564521+0000","last_fullsized":"2024-09-19T09:30:43.564521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.182130+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:30:52.182130+0000","last_peered":"2024-09-19T09:30:52.182130+0000","last_clean":"2024-09-19T09:30:52.182130+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:30:52.182130+0000","last_undegraded":"2024-09-19T09:30:52.182130+0000","last_fullsized":"2024-09-19T09:30:52.182130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562134+0000","last_change":"2024-09-19T09:30:42.563614+0000","last_active":"2024-09-19T09:30:43.562134+0000","last_peered":"2024-09-19T09:30:43.562134+0000","last_clean":"2024-09-19T09:30:43.562134+0000","last_became_active":"2024-09-19T09:30:42.563289+0000","last_became_peered":"2024-09-19T09:30:42.563289+0000","last_unstale":"2024-09-19T09:30:43.562134+0000","last_undegraded":"2024-09-19T09:30:43.562134+0000","last_fullsized":"2024-09-19T09:30:43.562134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:47.215006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":147,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.265965+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:31:38.265965+0000","last_peered":"2024-09-19T09:31:38.265965+0000","last_clean":"2024-09-19T09:31:38.265965+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:31:38.265965+0000","last_undegraded":"2024-09-19T09:31:38.265965+0000","last_fullsized":"2024-09-19T09:31:38.265965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":107,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561453+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:30:43.561453+0000","last_peered":"2024-09-19T09:30:43.561453+0000","last_clean":"2024-09-19T09:30:43.561453+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:30:43.561453+0000","last_undegraded":"2024-09-19T09:30:43.561453+0000","last_fullsized":"2024-09-19T09:30:43.561453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.191692+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:30:54.191692+0000","last_peered":"2024-09-19T09:30:54.191692+0000","last_clean":"2024-09-19T09:30:54.191692+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:30:54.191692+0000","last_undegraded":"2024-09-19T09:30:54.191692+0000","last_fullsized":"2024-09-19T09:30:54.191692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561229+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:30:43.561229+0000","last_peered":"2024-09-19T09:30:43.561229+0000","last_clean":"2024-09-19T09:30:43.561229+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:30:43.561229+0000","last_undegraded":"2024-09-19T09:30:43.561229+0000","last_fullsized":"2024-09-19T09:30:43.561229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188935+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:30:53.188935+0000","last_peered":"2024-09-19T09:30:53.188935+0000","last_clean":"2024-09-19T09:30:53.188935+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:30:53.188935+0000","last_undegraded":"2024-09-19T09:30:53.188935+0000","last_fullsized":"2024-09-19T09:30:53.188935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564099+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:30:43.564099+0000","last_peered":"2024-09-19T09:30:43.564099+0000","last_clean":"2024-09-19T09:30:43.564099+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:30:43.564099+0000","last_undegraded":"2024-09-19T09:30:43.564099+0000","last_fullsized":"2024-09-19T09:30:43.564099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":70,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:35.249315+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:31:35.249315+0000","last_peered":"2024-09-19T09:31:35.249315+0000","last_clean":"2024-09-19T09:31:35.249315+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:31:35.249315+0000","last_undegraded":"2024-09-19T09:31:35.249315+0000","last_fullsized":"2024-09-19T09:31:35.249315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562555+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:30:43.562555+0000","last_peered":"2024-09-19T09:30:43.562555+0000","last_clean":"2024-09-19T09:30:43.562555+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:30:43.562555+0000","last_undegraded":"2024-09-19T09:30:43.562555+0000","last_fullsized":"2024-09-19T09:30:43.562555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186229+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:30:53.186229+0000","last_peered":"2024-09-19T09:30:53.186229+0000","last_clean":"2024-09-19T09:30:53.186229+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:30:53.186229+0000","last_undegraded":"2024-09-19T09:30:53.186229+0000","last_fullsized":"2024-09-19T09:30:53.186229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544131+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:30:43.544131+0000","last_peered":"2024-09-19T09:30:43.544131+0000","last_clean":"2024-09-19T09:30:43.544131+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:30:43.544131+0000","last_undegraded":"2024-09-19T09:30:43.544131+0000","last_fullsized":"2024-09-19T09:30:43.544131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.160939+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:30:53.160939+0000","last_peered":"2024-09-19T09:30:53.160939+0000","last_clean":"2024-09-19T09:30:53.160939+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:30:53.160939+0000","last_undegraded":"2024-09-19T09:30:53.160939+0000","last_fullsized":"2024-09-19T09:30:53.160939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564588+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:30:43.564588+0000","last_peered":"2024-09-19T09:30:43.564588+0000","last_clean":"2024-09-19T09:30:43.564588+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:30:43.564588+0000","last_undegraded":"2024-09-19T09:30:43.564588+0000","last_fullsized":"2024-09-19T09:30:43.564588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186972+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:30:53.186972+0000","last_peered":"2024-09-19T09:30:53.186972+0000","last_clean":"2024-09-19T09:30:53.186972+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:30:53.186972+0000","last_undegraded":"2024-09-19T09:30:53.186972+0000","last_fullsized":"2024-09-19T09:30:53.186972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564675+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:30:43.564675+0000","last_peered":"2024-09-19T09:30:43.564675+0000","last_clean":"2024-09-19T09:30:43.564675+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:30:43.564675+0000","last_undegraded":"2024-09-19T09:30:43.564675+0000","last_fullsized":"2024-09-19T09:30:43.564675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186823+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:30:53.186823+0000","last_peered":"2024-09-19T09:30:53.186823+0000","last_clean":"2024-09-19T09:30:53.186823+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:30:53.186823+0000","last_undegraded":"2024-09-19T09:30:53.186823+0000","last_fullsized":"2024-09-19T09:30:53.186823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564388+0000","last_change":"2024-09-19T09:30:42.558972+0000","last_active":"2024-09-19T09:30:43.564388+0000","last_peered":"2024-09-19T09:30:43.564388+0000","last_clean":"2024-09-19T09:30:43.564388+0000","last_became_active":"2024-09-19T09:30:42.558388+0000","last_became_peered":"2024-09-19T09:30:42.558388+0000","last_unstale":"2024-09-19T09:30:43.564388+0000","last_undegraded":"2024-09-19T09:30:43.564388+0000","last_fullsized":"2024-09-19T09:30:43.564388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:20.537069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.261453+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:31:38.261453+0000","last_peered":"2024-09-19T09:31:38.261453+0000","last_clean":"2024-09-19T09:31:38.261453+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:31:38.261453+0000","last_undegraded":"2024-09-19T09:31:38.261453+0000","last_fullsized":"2024-09-19T09:31:38.261453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564607+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:30:43.564607+0000","last_peered":"2024-09-19T09:30:43.564607+0000","last_clean":"2024-09-19T09:30:43.564607+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:30:43.564607+0000","last_undegraded":"2024-09-19T09:30:43.564607+0000","last_fullsized":"2024-09-19T09:30:43.564607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562904+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:30:43.562904+0000","last_peered":"2024-09-19T09:30:43.562904+0000","last_clean":"2024-09-19T09:30:43.562904+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:30:43.562904+0000","last_undegraded":"2024-09-19T09:30:43.562904+0000","last_fullsized":"2024-09-19T09:30:43.562904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192354+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:30:54.192354+0000","last_peered":"2024-09-19T09:30:54.192354+0000","last_clean":"2024-09-19T09:30:54.192354+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:30:54.192354+0000","last_undegraded":"2024-09-19T09:30:54.192354+0000","last_fullsized":"2024-09-19T09:30:54.192354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542792+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:30:43.542792+0000","last_peered":"2024-09-19T09:30:43.542792+0000","last_clean":"2024-09-19T09:30:43.542792+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:30:43.542792+0000","last_undegraded":"2024-09-19T09:30:43.542792+0000","last_fullsized":"2024-09-19T09:30:43.542792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.165255+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:30:51.165255+0000","last_peered":"2024-09-19T09:30:51.165255+0000","last_clean":"2024-09-19T09:30:51.165255+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:30:51.165255+0000","last_undegraded":"2024-09-19T09:30:51.165255+0000","last_fullsized":"2024-09-19T09:30:51.165255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561146+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:30:43.561146+0000","last_peered":"2024-09-19T09:30:43.561146+0000","last_clean":"2024-09-19T09:30:43.561146+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:30:43.561146+0000","last_undegraded":"2024-09-19T09:30:43.561146+0000","last_fullsized":"2024-09-19T09:30:43.561146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:36.250154+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:31:36.250154+0000","last_peered":"2024-09-19T09:31:36.250154+0000","last_clean":"2024-09-19T09:31:36.250154+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:31:36.250154+0000","last_undegraded":"2024-09-19T09:31:36.250154+0000","last_fullsized":"2024-09-19T09:31:36.250154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543968+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:30:43.543968+0000","last_peered":"2024-09-19T09:30:43.543968+0000","last_clean":"2024-09-19T09:30:43.543968+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:30:43.543968+0000","last_undegraded":"2024-09-19T09:30:43.543968+0000","last_fullsized":"2024-09-19T09:30:43.543968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":69,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:35.249174+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:31:35.249174+0000","last_peered":"2024-09-19T09:31:35.249174+0000","last_clean":"2024-09-19T09:31:35.249174+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:31:35.249174+0000","last_undegraded":"2024-09-19T09:31:35.249174+0000","last_fullsized":"2024-09-19T09:31:35.249174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548405+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:30:43.548405+0000","last_peered":"2024-09-19T09:30:43.548405+0000","last_clean":"2024-09-19T09:30:43.548405+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:30:43.548405+0000","last_undegraded":"2024-09-19T09:30:43.548405+0000","last_fullsized":"2024-09-19T09:30:43.548405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560204+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:30:43.560204+0000","last_peered":"2024-09-19T09:30:43.560204+0000","last_clean":"2024-09-19T09:30:43.560204+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:30:43.560204+0000","last_undegraded":"2024-09-19T09:30:43.560204+0000","last_fullsized":"2024-09-19T09:30:43.560204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:44.011551+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:30:44.011551+0000","last_peered":"2024-09-19T09:30:44.011551+0000","last_clean":"2024-09-19T09:30:44.011551+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:30:44.011551+0000","last_undegraded":"2024-09-19T09:30:44.011551+0000","last_fullsized":"2024-09-19T09:30:44.011551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188733+0000","last_change":"2024-09-19T09:30:40.592827+0000","last_active":"2024-09-19T09:30:53.188733+0000","last_peered":"2024-09-19T09:30:53.188733+0000","last_clean":"2024-09-19T09:30:53.188733+0000","last_became_active":"2024-09-19T09:30:40.591986+0000","last_became_peered":"2024-09-19T09:30:40.591986+0000","last_unstale":"2024-09-19T09:30:53.188733+0000","last_undegraded":"2024-09-19T09:30:53.188733+0000","last_fullsized":"2024-09-19T09:30:53.188733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:11.426185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562237+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:30:43.562237+0000","last_peered":"2024-09-19T09:30:43.562237+0000","last_clean":"2024-09-19T09:30:43.562237+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:30:43.562237+0000","last_undegraded":"2024-09-19T09:30:43.562237+0000","last_fullsized":"2024-09-19T09:30:43.562237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:06.562377+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:31:06.562377+0000","last_peered":"2024-09-19T09:31:06.562377+0000","last_clean":"2024-09-19T09:31:06.562377+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:31:06.562377+0000","last_undegraded":"2024-09-19T09:31:06.562377+0000","last_fullsized":"2024-09-19T09:31:06.562377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543123+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:30:43.543123+0000","last_peered":"2024-09-19T09:30:43.543123+0000","last_clean":"2024-09-19T09:30:43.543123+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:30:43.543123+0000","last_undegraded":"2024-09-19T09:30:43.543123+0000","last_fullsized":"2024-09-19T09:30:43.543123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186347+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:30:53.186347+0000","last_peered":"2024-09-19T09:30:53.186347+0000","last_clean":"2024-09-19T09:30:53.186347+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:30:53.186347+0000","last_undegraded":"2024-09-19T09:30:53.186347+0000","last_fullsized":"2024-09-19T09:30:53.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564238+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:30:43.564238+0000","last_peered":"2024-09-19T09:30:43.564238+0000","last_clean":"2024-09-19T09:30:43.564238+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:30:43.564238+0000","last_undegraded":"2024-09-19T09:30:43.564238+0000","last_fullsized":"2024-09-19T09:30:43.564238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155515+0000","last_change":"2024-09-19T09:30:40.592013+0000","last_active":"2024-09-19T09:30:50.155515+0000","last_peered":"2024-09-19T09:30:50.155515+0000","last_clean":"2024-09-19T09:30:50.155515+0000","last_became_active":"2024-09-19T09:30:40.591742+0000","last_became_peered":"2024-09-19T09:30:40.591742+0000","last_unstale":"2024-09-19T09:30:50.155515+0000","last_undegraded":"2024-09-19T09:30:50.155515+0000","last_fullsized":"2024-09-19T09:30:50.155515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:24.014799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560917+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:30:43.560917+0000","last_peered":"2024-09-19T09:30:43.560917+0000","last_clean":"2024-09-19T09:30:43.560917+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:30:43.560917+0000","last_undegraded":"2024-09-19T09:30:43.560917+0000","last_fullsized":"2024-09-19T09:30:43.560917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186945+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:30:53.186945+0000","last_peered":"2024-09-19T09:30:53.186945+0000","last_clean":"2024-09-19T09:30:53.186945+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:30:53.186945+0000","last_undegraded":"2024-09-19T09:30:53.186945+0000","last_fullsized":"2024-09-19T09:30:53.186945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564140+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:30:43.564140+0000","last_peered":"2024-09-19T09:30:43.564140+0000","last_clean":"2024-09-19T09:30:43.564140+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:30:43.564140+0000","last_undegraded":"2024-09-19T09:30:43.564140+0000","last_fullsized":"2024-09-19T09:30:43.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184164+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:30:52.184164+0000","last_peered":"2024-09-19T09:30:52.184164+0000","last_clean":"2024-09-19T09:30:52.184164+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:30:52.184164+0000","last_undegraded":"2024-09-19T09:30:52.184164+0000","last_fullsized":"2024-09-19T09:30:52.184164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542815+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:30:43.542815+0000","last_peered":"2024-09-19T09:30:43.542815+0000","last_clean":"2024-09-19T09:30:43.542815+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:30:43.542815+0000","last_undegraded":"2024-09-19T09:30:43.542815+0000","last_fullsized":"2024-09-19T09:30:43.542815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155146+0000","last_change":"2024-09-19T09:30:40.535163+0000","last_active":"2024-09-19T09:30:50.155146+0000","last_peered":"2024-09-19T09:30:50.155146+0000","last_clean":"2024-09-19T09:30:50.155146+0000","last_became_active":"2024-09-19T09:30:40.534756+0000","last_became_peered":"2024-09-19T09:30:40.534756+0000","last_unstale":"2024-09-19T09:30:50.155146+0000","last_undegraded":"2024-09-19T09:30:50.155146+0000","last_fullsized":"2024-09-19T09:30:50.155146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:12:15.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563284+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:30:43.563284+0000","last_peered":"2024-09-19T09:30:43.563284+0000","last_clean":"2024-09-19T09:30:43.563284+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:30:43.563284+0000","last_undegraded":"2024-09-19T09:30:43.563284+0000","last_fullsized":"2024-09-19T09:30:43.563284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544223+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:30:43.544223+0000","last_peered":"2024-09-19T09:30:43.544223+0000","last_clean":"2024-09-19T09:30:43.544223+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:30:43.544223+0000","last_undegraded":"2024-09-19T09:30:43.544223+0000","last_fullsized":"2024-09-19T09:30:43.544223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188158+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:30:53.188158+0000","last_peered":"2024-09-19T09:30:53.188158+0000","last_clean":"2024-09-19T09:30:53.188158+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:30:53.188158+0000","last_undegraded":"2024-09-19T09:30:53.188158+0000","last_fullsized":"2024-09-19T09:30:53.188158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561983+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:30:43.561983+0000","last_peered":"2024-09-19T09:30:43.561983+0000","last_clean":"2024-09-19T09:30:43.561983+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:30:43.561983+0000","last_undegraded":"2024-09-19T09:30:43.561983+0000","last_fullsized":"2024-09-19T09:30:43.561983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192891+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:30:54.192891+0000","last_peered":"2024-09-19T09:30:54.192891+0000","last_clean":"2024-09-19T09:30:54.192891+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:30:54.192891+0000","last_undegraded":"2024-09-19T09:30:54.192891+0000","last_fullsized":"2024-09-19T09:30:54.192891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564534+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:30:43.564534+0000","last_peered":"2024-09-19T09:30:43.564534+0000","last_clean":"2024-09-19T09:30:43.564534+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:30:43.564534+0000","last_undegraded":"2024-09-19T09:30:43.564534+0000","last_fullsized":"2024-09-19T09:30:43.564534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.568190+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:30:43.568190+0000","last_peered":"2024-09-19T09:30:43.568190+0000","last_clean":"2024-09-19T09:30:43.568190+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:30:43.568190+0000","last_undegraded":"2024-09-19T09:30:43.568190+0000","last_fullsized":"2024-09-19T09:30:43.568190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543045+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:30:43.543045+0000","last_peered":"2024-09-19T09:30:43.543045+0000","last_clean":"2024-09-19T09:30:43.543045+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:30:43.543045+0000","last_undegraded":"2024-09-19T09:30:43.543045+0000","last_fullsized":"2024-09-19T09:30:43.543045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:37.251574+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:31:37.251574+0000","last_peered":"2024-09-19T09:31:37.251574+0000","last_clean":"2024-09-19T09:31:37.251574+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:31:37.251574+0000","last_undegraded":"2024-09-19T09:31:37.251574+0000","last_fullsized":"2024-09-19T09:31:37.251574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67365,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":490,"num_read_kb":319,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1066361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939683,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":892928,"data_stored":415105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":415483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1486848,"data_stored":1008789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1007681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":414647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":476112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":475892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:31:43.945 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:31:44.200 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:44.597 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:44 smithi028 ceph-mon[30972]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:44.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:44 smithi078 ceph-mon[35619]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:44.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:44 smithi038 ceph-mon[35271]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 -- 172.21.15.28:0/3375737921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 msgr2=0x7fc158100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 --2- 172.21.15.28:0/3375737921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158100ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc140009930 tx=0x7fc14002a420 comp rx=0 tx=0).stop 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 -- 172.21.15.28:0/3375737921 shutdown_connections 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 --2- 172.21.15.28:0/3375737921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc15810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 --2- 172.21.15.28:0/3375737921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158100ee0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 --2- 172.21.15.28:0/3375737921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 0x7fc158106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 -- 172.21.15.28:0/3375737921 >> 172.21.15.28:0/3375737921 conn(0x7fc1580fc7a0 msgr2=0x7fc1580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 -- 172.21.15.28:0/3375737921 shutdown_connections 2024-09-19T09:31:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 -- 172.21.15.28:0/3375737921 wait complete. 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.678+0000 7fc15ce94640 1 Processor -- start 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 -- start start 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc158199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 0x7fc1581a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc158110ec0 con 0x7fc158100a80 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc158110d40 con 0x7fc158106a80 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15ce94640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc158111040 con 0x7fc158101420 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43344/0 (socket says 172.21.15.28:43344) 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 -- 172.21.15.28:0/2526258232 learned_addr learned my addr 172.21.15.28:0/2526258232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc156d9d640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 0x7fc1581a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc14ffff640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc158199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 -- 172.21.15.28:0/2526258232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 msgr2=0x7fc158199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc158199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 -- 172.21.15.28:0/2526258232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 msgr2=0x7fc1581a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 0x7fc1581a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 -- 172.21.15.28:0/2526258232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1581a48e0 con 0x7fc158100a80 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc14ffff640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc158199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.679+0000 7fc15659c640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b4a0 tx=0x7fc14800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.680+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148014070 con 0x7fc158100a80 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.680+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc148010af0 con 0x7fc158100a80 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.680+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1581a4ab0 con 0x7fc158100a80 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.680+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148017d10 con 0x7fc158100a80 2024-09-19T09:31:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.680+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1581a4ff0 con 0x7fc158100a80 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.681+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc158100ee0 con 0x7fc158100a80 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.682+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc148017070 con 0x7fc158100a80 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.683+0000 7fc14f7fe640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 0x7fc1440a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.683+0000 7fc14ffff640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 0x7fc1440a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.683+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc1480cf090 con 0x7fc158100a80 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.683+0000 7fc14ffff640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 0x7fc1440a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc140000c00 tx=0x7fc140032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.686+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc14809be10 con 0x7fc158100a80 2024-09-19T09:31:44.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.858+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc158062630 con 0x7fc1440a3350 2024-09-19T09:31:44.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.864+0000 7fc14f7fe640 1 -- 172.21.15.28:0/2526258232 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258747 (secure 0 0 0) 0x7fc158062630 con 0x7fc1440a3350 2024-09-19T09:31:44.869 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 msgr2=0x7fc1440a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 0x7fc1440a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc140000c00 tx=0x7fc140032c70 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 msgr2=0x7fc158199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b4a0 tx=0x7fc14800b960 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 shutdown_connections 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc1440a3350 0x7fc1440a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc158106a80 0x7fc1581a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc158101420 0x7fc158199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 --2- 172.21.15.28:0/2526258232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc158100a80 0x7fc158199320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 >> 172.21.15.28:0/2526258232 conn(0x7fc1580fc7a0 msgr2=0x7fc1580fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 shutdown_connections 2024-09-19T09:31:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:44.867+0000 7fc15ce94640 1 -- 172.21.15.28:0/2526258232 wait complete. 2024-09-19T09:31:45.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:45 smithi028 ceph-mon[30972]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:31:45.589 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-19T09:31:43.409956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657967,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":636,"num_read_kb":438,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365756,"kb_used_data":12348,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576068,"statfs":{"total":1151940427776,"available":1151565893632,"internally_reserved":0,"allocated":12644352,"data_stored":6941697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003085"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565004+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:30:43.565004+0000","last_peered":"2024-09-19T09:30:43.565004+0000","last_clean":"2024-09-19T09:30:43.565004+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:30:43.565004+0000","last_undegraded":"2024-09-19T09:30:43.565004+0000","last_fullsized":"2024-09-19T09:30:43.565004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563865+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:30:43.563865+0000","last_peered":"2024-09-19T09:30:43.563865+0000","last_clean":"2024-09-19T09:30:43.563865+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:30:43.563865+0000","last_undegraded":"2024-09-19T09:30:43.563865+0000","last_fullsized":"2024-09-19T09:30:43.563865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564249+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:30:43.564249+0000","last_peered":"2024-09-19T09:30:43.564249+0000","last_clean":"2024-09-19T09:30:43.564249+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:30:43.564249+0000","last_undegraded":"2024-09-19T09:30:43.564249+0000","last_fullsized":"2024-09-19T09:30:43.564249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561198+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:30:43.561198+0000","last_peered":"2024-09-19T09:30:43.561198+0000","last_clean":"2024-09-19T09:30:43.561198+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:30:43.561198+0000","last_undegraded":"2024-09-19T09:30:43.561198+0000","last_fullsized":"2024-09-19T09:30:43.561198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561252+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:30:43.561252+0000","last_peered":"2024-09-19T09:30:43.561252+0000","last_clean":"2024-09-19T09:30:43.561252+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:30:43.561252+0000","last_undegraded":"2024-09-19T09:30:43.561252+0000","last_fullsized":"2024-09-19T09:30:43.561252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.547978+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:30:43.547978+0000","last_peered":"2024-09-19T09:30:43.547978+0000","last_clean":"2024-09-19T09:30:43.547978+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:30:43.547978+0000","last_undegraded":"2024-09-19T09:30:43.547978+0000","last_fullsized":"2024-09-19T09:30:43.547978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562172+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:30:43.562172+0000","last_peered":"2024-09-19T09:30:43.562172+0000","last_clean":"2024-09-19T09:30:43.562172+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:30:43.562172+0000","last_undegraded":"2024-09-19T09:30:43.562172+0000","last_fullsized":"2024-09-19T09:30:43.562172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542518+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:30:43.542518+0000","last_peered":"2024-09-19T09:30:43.542518+0000","last_clean":"2024-09-19T09:30:43.542518+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:30:43.542518+0000","last_undegraded":"2024-09-19T09:30:43.542518+0000","last_fullsized":"2024-09-19T09:30:43.542518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562779+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:30:43.562779+0000","last_peered":"2024-09-19T09:30:43.562779+0000","last_clean":"2024-09-19T09:30:43.562779+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:30:43.562779+0000","last_undegraded":"2024-09-19T09:30:43.562779+0000","last_fullsized":"2024-09-19T09:30:43.562779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548341+0000","last_change":"2024-09-19T09:30:42.564174+0000","last_active":"2024-09-19T09:30:43.548341+0000","last_peered":"2024-09-19T09:30:43.548341+0000","last_clean":"2024-09-19T09:30:43.548341+0000","last_became_active":"2024-09-19T09:30:42.563790+0000","last_became_peered":"2024-09-19T09:30:42.563790+0000","last_unstale":"2024-09-19T09:30:43.548341+0000","last_undegraded":"2024-09-19T09:30:43.548341+0000","last_fullsized":"2024-09-19T09:30:43.548341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:22.463896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564526+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:30:43.564526+0000","last_peered":"2024-09-19T09:30:43.564526+0000","last_clean":"2024-09-19T09:30:43.564526+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:30:43.564526+0000","last_undegraded":"2024-09-19T09:30:43.564526+0000","last_fullsized":"2024-09-19T09:30:43.564526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561947+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:30:43.561947+0000","last_peered":"2024-09-19T09:30:43.561947+0000","last_clean":"2024-09-19T09:30:43.561947+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:30:43.561947+0000","last_undegraded":"2024-09-19T09:30:43.561947+0000","last_fullsized":"2024-09-19T09:30:43.561947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561657+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:30:43.561657+0000","last_peered":"2024-09-19T09:30:43.561657+0000","last_clean":"2024-09-19T09:30:43.561657+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:30:43.561657+0000","last_undegraded":"2024-09-19T09:30:43.561657+0000","last_fullsized":"2024-09-19T09:30:43.561657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562189+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:30:43.562189+0000","last_peered":"2024-09-19T09:30:43.562189+0000","last_clean":"2024-09-19T09:30:43.562189+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:30:43.562189+0000","last_undegraded":"2024-09-19T09:30:43.562189+0000","last_fullsized":"2024-09-19T09:30:43.562189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562374+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:30:43.562374+0000","last_peered":"2024-09-19T09:30:43.562374+0000","last_clean":"2024-09-19T09:30:43.562374+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:30:43.562374+0000","last_undegraded":"2024-09-19T09:30:43.562374+0000","last_fullsized":"2024-09-19T09:30:43.562374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560910+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:30:43.560910+0000","last_peered":"2024-09-19T09:30:43.560910+0000","last_clean":"2024-09-19T09:30:43.560910+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:30:43.560910+0000","last_undegraded":"2024-09-19T09:30:43.560910+0000","last_fullsized":"2024-09-19T09:30:43.560910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565188+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:30:43.565188+0000","last_peered":"2024-09-19T09:30:43.565188+0000","last_clean":"2024-09-19T09:30:43.565188+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:30:43.565188+0000","last_undegraded":"2024-09-19T09:30:43.565188+0000","last_fullsized":"2024-09-19T09:30:43.565188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560190+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:30:43.560190+0000","last_peered":"2024-09-19T09:30:43.560190+0000","last_clean":"2024-09-19T09:30:43.560190+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:30:43.560190+0000","last_undegraded":"2024-09-19T09:30:43.560190+0000","last_fullsized":"2024-09-19T09:30:43.560190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563025+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:30:43.563025+0000","last_peered":"2024-09-19T09:30:43.563025+0000","last_clean":"2024-09-19T09:30:43.563025+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:30:43.563025+0000","last_undegraded":"2024-09-19T09:30:43.563025+0000","last_fullsized":"2024-09-19T09:30:43.563025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564597+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:30:43.564597+0000","last_peered":"2024-09-19T09:30:43.564597+0000","last_clean":"2024-09-19T09:30:43.564597+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:30:43.564597+0000","last_undegraded":"2024-09-19T09:30:43.564597+0000","last_fullsized":"2024-09-19T09:30:43.564597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561822+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:30:43.561822+0000","last_peered":"2024-09-19T09:30:43.561822+0000","last_clean":"2024-09-19T09:30:43.561822+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:30:43.561822+0000","last_undegraded":"2024-09-19T09:30:43.561822+0000","last_fullsized":"2024-09-19T09:30:43.561822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561430+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:30:43.561430+0000","last_peered":"2024-09-19T09:30:43.561430+0000","last_clean":"2024-09-19T09:30:43.561430+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:30:43.561430+0000","last_undegraded":"2024-09-19T09:30:43.561430+0000","last_fullsized":"2024-09-19T09:30:43.561430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542541+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:30:43.542541+0000","last_peered":"2024-09-19T09:30:43.542541+0000","last_clean":"2024-09-19T09:30:43.542541+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:30:43.542541+0000","last_undegraded":"2024-09-19T09:30:43.542541+0000","last_fullsized":"2024-09-19T09:30:43.542541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544194+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:30:43.544194+0000","last_peered":"2024-09-19T09:30:43.544194+0000","last_clean":"2024-09-19T09:30:43.544194+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:30:43.544194+0000","last_undegraded":"2024-09-19T09:30:43.544194+0000","last_fullsized":"2024-09-19T09:30:43.544194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542969+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:30:43.542969+0000","last_peered":"2024-09-19T09:30:43.542969+0000","last_clean":"2024-09-19T09:30:43.542969+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:30:43.542969+0000","last_undegraded":"2024-09-19T09:30:43.542969+0000","last_fullsized":"2024-09-19T09:30:43.542969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.565943+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:30:43.565943+0000","last_peered":"2024-09-19T09:30:43.565943+0000","last_clean":"2024-09-19T09:30:43.565943+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:30:43.565943+0000","last_undegraded":"2024-09-19T09:30:43.565943+0000","last_fullsized":"2024-09-19T09:30:43.565943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564893+0000","last_change":"2024-09-19T09:30:42.559055+0000","last_active":"2024-09-19T09:30:43.564893+0000","last_peered":"2024-09-19T09:30:43.564893+0000","last_clean":"2024-09-19T09:30:43.564893+0000","last_became_active":"2024-09-19T09:30:42.558537+0000","last_became_peered":"2024-09-19T09:30:42.558537+0000","last_unstale":"2024-09-19T09:30:43.564893+0000","last_undegraded":"2024-09-19T09:30:43.564893+0000","last_fullsized":"2024-09-19T09:30:43.564893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.777026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562079+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:30:43.562079+0000","last_peered":"2024-09-19T09:30:43.562079+0000","last_clean":"2024-09-19T09:30:43.562079+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:30:43.562079+0000","last_undegraded":"2024-09-19T09:30:43.562079+0000","last_fullsized":"2024-09-19T09:30:43.562079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564360+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:30:43.564360+0000","last_peered":"2024-09-19T09:30:43.564360+0000","last_clean":"2024-09-19T09:30:43.564360+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:30:43.564360+0000","last_undegraded":"2024-09-19T09:30:43.564360+0000","last_fullsized":"2024-09-19T09:30:43.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562024+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:30:43.562024+0000","last_peered":"2024-09-19T09:30:43.562024+0000","last_clean":"2024-09-19T09:30:43.562024+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:30:43.562024+0000","last_undegraded":"2024-09-19T09:30:43.562024+0000","last_fullsized":"2024-09-19T09:30:43.562024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563172+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:30:43.563172+0000","last_peered":"2024-09-19T09:30:43.563172+0000","last_clean":"2024-09-19T09:30:43.563172+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:30:43.563172+0000","last_undegraded":"2024-09-19T09:30:43.563172+0000","last_fullsized":"2024-09-19T09:30:43.563172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563982+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:30:43.563982+0000","last_peered":"2024-09-19T09:30:43.563982+0000","last_clean":"2024-09-19T09:30:43.563982+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:30:43.563982+0000","last_undegraded":"2024-09-19T09:30:43.563982+0000","last_fullsized":"2024-09-19T09:30:43.563982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560987+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:30:43.560987+0000","last_peered":"2024-09-19T09:30:43.560987+0000","last_clean":"2024-09-19T09:30:43.560987+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:30:43.560987+0000","last_undegraded":"2024-09-19T09:30:43.560987+0000","last_fullsized":"2024-09-19T09:30:43.560987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:39.255347+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:31:39.255347+0000","last_peered":"2024-09-19T09:31:39.255347+0000","last_clean":"2024-09-19T09:31:39.255347+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:31:39.255347+0000","last_undegraded":"2024-09-19T09:31:39.255347+0000","last_fullsized":"2024-09-19T09:31:39.255347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544051+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:30:43.544051+0000","last_peered":"2024-09-19T09:30:43.544051+0000","last_clean":"2024-09-19T09:30:43.544051+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:30:43.544051+0000","last_undegraded":"2024-09-19T09:30:43.544051+0000","last_fullsized":"2024-09-19T09:30:43.544051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184065+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:30:52.184065+0000","last_peered":"2024-09-19T09:30:52.184065+0000","last_clean":"2024-09-19T09:30:52.184065+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:30:52.184065+0000","last_undegraded":"2024-09-19T09:30:52.184065+0000","last_fullsized":"2024-09-19T09:30:52.184065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.163943+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:30:51.163943+0000","last_peered":"2024-09-19T09:30:51.163943+0000","last_clean":"2024-09-19T09:30:51.163943+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:30:51.163943+0000","last_undegraded":"2024-09-19T09:30:51.163943+0000","last_fullsized":"2024-09-19T09:30:51.163943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542591+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:30:43.542591+0000","last_peered":"2024-09-19T09:30:43.542591+0000","last_clean":"2024-09-19T09:30:43.542591+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:30:43.542591+0000","last_undegraded":"2024-09-19T09:30:43.542591+0000","last_fullsized":"2024-09-19T09:30:43.542591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187883+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:30:53.187883+0000","last_peered":"2024-09-19T09:30:53.187883+0000","last_clean":"2024-09-19T09:30:53.187883+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:30:53.187883+0000","last_undegraded":"2024-09-19T09:30:53.187883+0000","last_fullsized":"2024-09-19T09:30:53.187883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.559954+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:30:43.559954+0000","last_peered":"2024-09-19T09:30:43.559954+0000","last_clean":"2024-09-19T09:30:43.559954+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:30:43.559954+0000","last_undegraded":"2024-09-19T09:30:43.559954+0000","last_fullsized":"2024-09-19T09:30:43.559954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":112,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.253519+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:31:38.253519+0000","last_peered":"2024-09-19T09:31:38.253519+0000","last_clean":"2024-09-19T09:31:38.253519+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:31:38.253519+0000","last_undegraded":"2024-09-19T09:31:38.253519+0000","last_fullsized":"2024-09-19T09:31:38.253519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562997+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:30:43.562997+0000","last_peered":"2024-09-19T09:30:43.562997+0000","last_clean":"2024-09-19T09:30:43.562997+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:30:43.562997+0000","last_undegraded":"2024-09-19T09:30:43.562997+0000","last_fullsized":"2024-09-19T09:30:43.562997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.187517+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:30:53.187517+0000","last_peered":"2024-09-19T09:30:53.187517+0000","last_clean":"2024-09-19T09:30:53.187517+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:30:53.187517+0000","last_undegraded":"2024-09-19T09:30:53.187517+0000","last_fullsized":"2024-09-19T09:30:53.187517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564521+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:30:43.564521+0000","last_peered":"2024-09-19T09:30:43.564521+0000","last_clean":"2024-09-19T09:30:43.564521+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:30:43.564521+0000","last_undegraded":"2024-09-19T09:30:43.564521+0000","last_fullsized":"2024-09-19T09:30:43.564521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.182130+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:30:52.182130+0000","last_peered":"2024-09-19T09:30:52.182130+0000","last_clean":"2024-09-19T09:30:52.182130+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:30:52.182130+0000","last_undegraded":"2024-09-19T09:30:52.182130+0000","last_fullsized":"2024-09-19T09:30:52.182130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562134+0000","last_change":"2024-09-19T09:30:42.563614+0000","last_active":"2024-09-19T09:30:43.562134+0000","last_peered":"2024-09-19T09:30:43.562134+0000","last_clean":"2024-09-19T09:30:43.562134+0000","last_became_active":"2024-09-19T09:30:42.563289+0000","last_became_peered":"2024-09-19T09:30:42.563289+0000","last_unstale":"2024-09-19T09:30:43.562134+0000","last_undegraded":"2024-09-19T09:30:43.562134+0000","last_fullsized":"2024-09-19T09:30:43.562134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:47.215006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":147,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.265965+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:31:38.265965+0000","last_peered":"2024-09-19T09:31:38.265965+0000","last_clean":"2024-09-19T09:31:38.265965+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:31:38.265965+0000","last_undegraded":"2024-09-19T09:31:38.265965+0000","last_fullsized":"2024-09-19T09:31:38.265965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":107,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561453+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:30:43.561453+0000","last_peered":"2024-09-19T09:30:43.561453+0000","last_clean":"2024-09-19T09:30:43.561453+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:30:43.561453+0000","last_undegraded":"2024-09-19T09:30:43.561453+0000","last_fullsized":"2024-09-19T09:30:43.561453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.191692+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:30:54.191692+0000","last_peered":"2024-09-19T09:30:54.191692+0000","last_clean":"2024-09-19T09:30:54.191692+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:30:54.191692+0000","last_undegraded":"2024-09-19T09:30:54.191692+0000","last_fullsized":"2024-09-19T09:30:54.191692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561229+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:30:43.561229+0000","last_peered":"2024-09-19T09:30:43.561229+0000","last_clean":"2024-09-19T09:30:43.561229+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:30:43.561229+0000","last_undegraded":"2024-09-19T09:30:43.561229+0000","last_fullsized":"2024-09-19T09:30:43.561229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188935+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:30:53.188935+0000","last_peered":"2024-09-19T09:30:53.188935+0000","last_clean":"2024-09-19T09:30:53.188935+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:30:53.188935+0000","last_undegraded":"2024-09-19T09:30:53.188935+0000","last_fullsized":"2024-09-19T09:30:53.188935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564099+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:30:43.564099+0000","last_peered":"2024-09-19T09:30:43.564099+0000","last_clean":"2024-09-19T09:30:43.564099+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:30:43.564099+0000","last_undegraded":"2024-09-19T09:30:43.564099+0000","last_fullsized":"2024-09-19T09:30:43.564099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:40.256150+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:31:40.256150+0000","last_peered":"2024-09-19T09:31:40.256150+0000","last_clean":"2024-09-19T09:31:40.256150+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:31:40.256150+0000","last_undegraded":"2024-09-19T09:31:40.256150+0000","last_fullsized":"2024-09-19T09:31:40.256150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562555+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:30:43.562555+0000","last_peered":"2024-09-19T09:30:43.562555+0000","last_clean":"2024-09-19T09:30:43.562555+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:30:43.562555+0000","last_undegraded":"2024-09-19T09:30:43.562555+0000","last_fullsized":"2024-09-19T09:30:43.562555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186229+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:30:53.186229+0000","last_peered":"2024-09-19T09:30:53.186229+0000","last_clean":"2024-09-19T09:30:53.186229+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:30:53.186229+0000","last_undegraded":"2024-09-19T09:30:53.186229+0000","last_fullsized":"2024-09-19T09:30:53.186229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544131+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:30:43.544131+0000","last_peered":"2024-09-19T09:30:43.544131+0000","last_clean":"2024-09-19T09:30:43.544131+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:30:43.544131+0000","last_undegraded":"2024-09-19T09:30:43.544131+0000","last_fullsized":"2024-09-19T09:30:43.544131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.160939+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:30:53.160939+0000","last_peered":"2024-09-19T09:30:53.160939+0000","last_clean":"2024-09-19T09:30:53.160939+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:30:53.160939+0000","last_undegraded":"2024-09-19T09:30:53.160939+0000","last_fullsized":"2024-09-19T09:30:53.160939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564588+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:30:43.564588+0000","last_peered":"2024-09-19T09:30:43.564588+0000","last_clean":"2024-09-19T09:30:43.564588+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:30:43.564588+0000","last_undegraded":"2024-09-19T09:30:43.564588+0000","last_fullsized":"2024-09-19T09:30:43.564588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186972+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:30:53.186972+0000","last_peered":"2024-09-19T09:30:53.186972+0000","last_clean":"2024-09-19T09:30:53.186972+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:30:53.186972+0000","last_undegraded":"2024-09-19T09:30:53.186972+0000","last_fullsized":"2024-09-19T09:30:53.186972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564675+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:30:43.564675+0000","last_peered":"2024-09-19T09:30:43.564675+0000","last_clean":"2024-09-19T09:30:43.564675+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:30:43.564675+0000","last_undegraded":"2024-09-19T09:30:43.564675+0000","last_fullsized":"2024-09-19T09:30:43.564675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186823+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:30:53.186823+0000","last_peered":"2024-09-19T09:30:53.186823+0000","last_clean":"2024-09-19T09:30:53.186823+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:30:53.186823+0000","last_undegraded":"2024-09-19T09:30:53.186823+0000","last_fullsized":"2024-09-19T09:30:53.186823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564388+0000","last_change":"2024-09-19T09:30:42.558972+0000","last_active":"2024-09-19T09:30:43.564388+0000","last_peered":"2024-09-19T09:30:43.564388+0000","last_clean":"2024-09-19T09:30:43.564388+0000","last_became_active":"2024-09-19T09:30:42.558388+0000","last_became_peered":"2024-09-19T09:30:42.558388+0000","last_unstale":"2024-09-19T09:30:43.564388+0000","last_undegraded":"2024-09-19T09:30:43.564388+0000","last_fullsized":"2024-09-19T09:30:43.564388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:20.537069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:38.261453+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:31:38.261453+0000","last_peered":"2024-09-19T09:31:38.261453+0000","last_clean":"2024-09-19T09:31:38.261453+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:31:38.261453+0000","last_undegraded":"2024-09-19T09:31:38.261453+0000","last_fullsized":"2024-09-19T09:31:38.261453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564607+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:30:43.564607+0000","last_peered":"2024-09-19T09:30:43.564607+0000","last_clean":"2024-09-19T09:30:43.564607+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:30:43.564607+0000","last_undegraded":"2024-09-19T09:30:43.564607+0000","last_fullsized":"2024-09-19T09:30:43.564607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562904+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:30:43.562904+0000","last_peered":"2024-09-19T09:30:43.562904+0000","last_clean":"2024-09-19T09:30:43.562904+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:30:43.562904+0000","last_undegraded":"2024-09-19T09:30:43.562904+0000","last_fullsized":"2024-09-19T09:30:43.562904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192354+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:30:54.192354+0000","last_peered":"2024-09-19T09:30:54.192354+0000","last_clean":"2024-09-19T09:30:54.192354+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:30:54.192354+0000","last_undegraded":"2024-09-19T09:30:54.192354+0000","last_fullsized":"2024-09-19T09:30:54.192354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542792+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:30:43.542792+0000","last_peered":"2024-09-19T09:30:43.542792+0000","last_clean":"2024-09-19T09:30:43.542792+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:30:43.542792+0000","last_undegraded":"2024-09-19T09:30:43.542792+0000","last_fullsized":"2024-09-19T09:30:43.542792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:51.165255+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:30:51.165255+0000","last_peered":"2024-09-19T09:30:51.165255+0000","last_clean":"2024-09-19T09:30:51.165255+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:30:51.165255+0000","last_undegraded":"2024-09-19T09:30:51.165255+0000","last_fullsized":"2024-09-19T09:30:51.165255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561146+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:30:43.561146+0000","last_peered":"2024-09-19T09:30:43.561146+0000","last_clean":"2024-09-19T09:30:43.561146+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:30:43.561146+0000","last_undegraded":"2024-09-19T09:30:43.561146+0000","last_fullsized":"2024-09-19T09:30:43.561146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":73,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:41.257044+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:31:41.257044+0000","last_peered":"2024-09-19T09:31:41.257044+0000","last_clean":"2024-09-19T09:31:41.257044+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:31:41.257044+0000","last_undegraded":"2024-09-19T09:31:41.257044+0000","last_fullsized":"2024-09-19T09:31:41.257044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543968+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:30:43.543968+0000","last_peered":"2024-09-19T09:30:43.543968+0000","last_clean":"2024-09-19T09:30:43.543968+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:30:43.543968+0000","last_undegraded":"2024-09-19T09:30:43.543968+0000","last_fullsized":"2024-09-19T09:30:43.543968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:40.256088+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:31:40.256088+0000","last_peered":"2024-09-19T09:31:40.256088+0000","last_clean":"2024-09-19T09:31:40.256088+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:31:40.256088+0000","last_undegraded":"2024-09-19T09:31:40.256088+0000","last_fullsized":"2024-09-19T09:31:40.256088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.548405+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:30:43.548405+0000","last_peered":"2024-09-19T09:30:43.548405+0000","last_clean":"2024-09-19T09:30:43.548405+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:30:43.548405+0000","last_undegraded":"2024-09-19T09:30:43.548405+0000","last_fullsized":"2024-09-19T09:30:43.548405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560204+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:30:43.560204+0000","last_peered":"2024-09-19T09:30:43.560204+0000","last_clean":"2024-09-19T09:30:43.560204+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:30:43.560204+0000","last_undegraded":"2024-09-19T09:30:43.560204+0000","last_fullsized":"2024-09-19T09:30:43.560204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:44.011551+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:30:44.011551+0000","last_peered":"2024-09-19T09:30:44.011551+0000","last_clean":"2024-09-19T09:30:44.011551+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:30:44.011551+0000","last_undegraded":"2024-09-19T09:30:44.011551+0000","last_fullsized":"2024-09-19T09:30:44.011551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188733+0000","last_change":"2024-09-19T09:30:40.592827+0000","last_active":"2024-09-19T09:30:53.188733+0000","last_peered":"2024-09-19T09:30:53.188733+0000","last_clean":"2024-09-19T09:30:53.188733+0000","last_became_active":"2024-09-19T09:30:40.591986+0000","last_became_peered":"2024-09-19T09:30:40.591986+0000","last_unstale":"2024-09-19T09:30:53.188733+0000","last_undegraded":"2024-09-19T09:30:53.188733+0000","last_fullsized":"2024-09-19T09:30:53.188733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:11.426185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.562237+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:30:43.562237+0000","last_peered":"2024-09-19T09:30:43.562237+0000","last_clean":"2024-09-19T09:30:43.562237+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:30:43.562237+0000","last_undegraded":"2024-09-19T09:30:43.562237+0000","last_fullsized":"2024-09-19T09:30:43.562237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:06.562377+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:31:06.562377+0000","last_peered":"2024-09-19T09:31:06.562377+0000","last_clean":"2024-09-19T09:31:06.562377+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:31:06.562377+0000","last_undegraded":"2024-09-19T09:31:06.562377+0000","last_fullsized":"2024-09-19T09:31:06.562377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543123+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:30:43.543123+0000","last_peered":"2024-09-19T09:30:43.543123+0000","last_clean":"2024-09-19T09:30:43.543123+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:30:43.543123+0000","last_undegraded":"2024-09-19T09:30:43.543123+0000","last_fullsized":"2024-09-19T09:30:43.543123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186347+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:30:53.186347+0000","last_peered":"2024-09-19T09:30:53.186347+0000","last_clean":"2024-09-19T09:30:53.186347+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:30:53.186347+0000","last_undegraded":"2024-09-19T09:30:53.186347+0000","last_fullsized":"2024-09-19T09:30:53.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564238+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:30:43.564238+0000","last_peered":"2024-09-19T09:30:43.564238+0000","last_clean":"2024-09-19T09:30:43.564238+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:30:43.564238+0000","last_undegraded":"2024-09-19T09:30:43.564238+0000","last_fullsized":"2024-09-19T09:30:43.564238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155515+0000","last_change":"2024-09-19T09:30:40.592013+0000","last_active":"2024-09-19T09:30:50.155515+0000","last_peered":"2024-09-19T09:30:50.155515+0000","last_clean":"2024-09-19T09:30:50.155515+0000","last_became_active":"2024-09-19T09:30:40.591742+0000","last_became_peered":"2024-09-19T09:30:40.591742+0000","last_unstale":"2024-09-19T09:30:50.155515+0000","last_undegraded":"2024-09-19T09:30:50.155515+0000","last_fullsized":"2024-09-19T09:30:50.155515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:24.014799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.560917+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:30:43.560917+0000","last_peered":"2024-09-19T09:30:43.560917+0000","last_clean":"2024-09-19T09:30:43.560917+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:30:43.560917+0000","last_undegraded":"2024-09-19T09:30:43.560917+0000","last_fullsized":"2024-09-19T09:30:43.560917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.186945+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:30:53.186945+0000","last_peered":"2024-09-19T09:30:53.186945+0000","last_clean":"2024-09-19T09:30:53.186945+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:30:53.186945+0000","last_undegraded":"2024-09-19T09:30:53.186945+0000","last_fullsized":"2024-09-19T09:30:53.186945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564140+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:30:43.564140+0000","last_peered":"2024-09-19T09:30:43.564140+0000","last_clean":"2024-09-19T09:30:43.564140+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:30:43.564140+0000","last_undegraded":"2024-09-19T09:30:43.564140+0000","last_fullsized":"2024-09-19T09:30:43.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:52.184164+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:30:52.184164+0000","last_peered":"2024-09-19T09:30:52.184164+0000","last_clean":"2024-09-19T09:30:52.184164+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:30:52.184164+0000","last_undegraded":"2024-09-19T09:30:52.184164+0000","last_fullsized":"2024-09-19T09:30:52.184164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.542815+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:30:43.542815+0000","last_peered":"2024-09-19T09:30:43.542815+0000","last_clean":"2024-09-19T09:30:43.542815+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:30:43.542815+0000","last_undegraded":"2024-09-19T09:30:43.542815+0000","last_fullsized":"2024-09-19T09:30:43.542815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:50.155146+0000","last_change":"2024-09-19T09:30:40.535163+0000","last_active":"2024-09-19T09:30:50.155146+0000","last_peered":"2024-09-19T09:30:50.155146+0000","last_clean":"2024-09-19T09:30:50.155146+0000","last_became_active":"2024-09-19T09:30:40.534756+0000","last_became_peered":"2024-09-19T09:30:40.534756+0000","last_unstale":"2024-09-19T09:30:50.155146+0000","last_undegraded":"2024-09-19T09:30:50.155146+0000","last_fullsized":"2024-09-19T09:30:50.155146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:12:15.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.563284+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:30:43.563284+0000","last_peered":"2024-09-19T09:30:43.563284+0000","last_clean":"2024-09-19T09:30:43.563284+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:30:43.563284+0000","last_undegraded":"2024-09-19T09:30:43.563284+0000","last_fullsized":"2024-09-19T09:30:43.563284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.544223+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:30:43.544223+0000","last_peered":"2024-09-19T09:30:43.544223+0000","last_clean":"2024-09-19T09:30:43.544223+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:30:43.544223+0000","last_undegraded":"2024-09-19T09:30:43.544223+0000","last_fullsized":"2024-09-19T09:30:43.544223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:53.188158+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:30:53.188158+0000","last_peered":"2024-09-19T09:30:53.188158+0000","last_clean":"2024-09-19T09:30:53.188158+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:30:53.188158+0000","last_undegraded":"2024-09-19T09:30:53.188158+0000","last_fullsized":"2024-09-19T09:30:53.188158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.561983+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:30:43.561983+0000","last_peered":"2024-09-19T09:30:43.561983+0000","last_clean":"2024-09-19T09:30:43.561983+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:30:43.561983+0000","last_undegraded":"2024-09-19T09:30:43.561983+0000","last_fullsized":"2024-09-19T09:30:43.561983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:54.192891+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:30:54.192891+0000","last_peered":"2024-09-19T09:30:54.192891+0000","last_clean":"2024-09-19T09:30:54.192891+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:30:54.192891+0000","last_undegraded":"2024-09-19T09:30:54.192891+0000","last_fullsized":"2024-09-19T09:30:54.192891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.564534+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:30:43.564534+0000","last_peered":"2024-09-19T09:30:43.564534+0000","last_clean":"2024-09-19T09:30:43.564534+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:30:43.564534+0000","last_undegraded":"2024-09-19T09:30:43.564534+0000","last_fullsized":"2024-09-19T09:30:43.564534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.568190+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:30:43.568190+0000","last_peered":"2024-09-19T09:30:43.568190+0000","last_clean":"2024-09-19T09:30:43.568190+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:30:43.568190+0000","last_undegraded":"2024-09-19T09:30:43.568190+0000","last_fullsized":"2024-09-19T09:30:43.568190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:30:43.543045+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:30:43.543045+0000","last_peered":"2024-09-19T09:30:43.543045+0000","last_clean":"2024-09-19T09:30:43.543045+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:30:43.543045+0000","last_undegraded":"2024-09-19T09:30:43.543045+0000","last_fullsized":"2024-09-19T09:30:43.543045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":82,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:31:42.258456+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:31:42.258456+0000","last_peered":"2024-09-19T09:31:42.258456+0000","last_clean":"2024-09-19T09:31:42.258456+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:31:42.258456+0000","last_undegraded":"2024-09-19T09:31:42.258456+0000","last_fullsized":"2024-09-19T09:31:42.258456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67365,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":510,"num_read_kb":329,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1066361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":892928,"data_stored":415105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":415483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712516,"statfs":{"total":95995035648,"available":95961616384,"internally_reserved":0,"allocated":1486848,"data_stored":1008789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1499136,"data_stored":1007681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":415076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":414647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":476112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":475892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:31:45.593 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:31:45.594 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T09:31:45.594 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T09:31:45.594 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T09:31:45.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:45 smithi078 ceph-mon[35619]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:31:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:45 smithi038 ceph-mon[35271]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:31:45.851 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:46.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 -- 172.21.15.28:0/2244115321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c103760 msgr2=0x7fd85c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 --2- 172.21.15.28:0/2244115321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c103760 0x7fd85c107ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd844009930 tx=0x7fd84402a420 comp rx=0 tx=0).stop 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 -- 172.21.15.28:0/2244115321 shutdown_connections 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 --2- 172.21.15.28:0/2244115321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c1082e0 0x7fd85c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 --2- 172.21.15.28:0/2244115321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c103760 0x7fd85c107ba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 --2- 172.21.15.28:0/2244115321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c102db0 0x7fd85c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 -- 172.21.15.28:0/2244115321 >> 172.21.15.28:0/2244115321 conn(0x7fd85c0fc7d0 msgr2=0x7fd85c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 -- 172.21.15.28:0/2244115321 shutdown_connections 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.259+0000 7fd862836640 1 -- 172.21.15.28:0/2244115321 wait complete. 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 Processor -- start 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 -- start start 2024-09-19T09:31:46.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c102db0 0x7fd85c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c103760 0x7fd85c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c112050 con 0x7fd85c1082e0 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c111ed0 con 0x7fd85c102db0 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd862836640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1121d0 con 0x7fd85c103760 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43366/0 (socket says 172.21.15.28:43366) 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 -- 172.21.15.28:0/3936199204 learned_addr learned my addr 172.21.15.28:0/3936199204 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd85bfff640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c102db0 0x7fd85c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd853fff640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c103760 0x7fd85c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 -- 172.21.15.28:0/3936199204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c103760 msgr2=0x7fd85c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c103760 0x7fd85c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 -- 172.21.15.28:0/3936199204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c102db0 msgr2=0x7fd85c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c102db0 0x7fd85c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.260+0000 7fd860dd3640 1 -- 172.21.15.28:0/3936199204 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c19d560 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd860dd3640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eaa0 tx=0x7fd85400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85400cdb0 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c19d850 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd854005020 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540116b0 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.261+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1a5130 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.262+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c103190 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.263+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd854005560 con 0x7fd85c1082e0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.264+0000 7fd859ffb640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 0x7fd8480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.264+0000 7fd85bfff640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 0x7fd8480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:46.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.264+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd8540cfa30 con 0x7fd85c1082e0 2024-09-19T09:31:46.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.264+0000 7fd85bfff640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 0x7fd8480a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0058c0 tx=0x7fd84c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:46.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.268+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd854016030 con 0x7fd85c1082e0 2024-09-19T09:31:46.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.481+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd85c10a2c0 con 0x7fd85c1082e0 2024-09-19T09:31:46.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.482+0000 7fd859ffb640 1 -- 172.21.15.28:0/3936199204 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd85409c150 con 0x7fd85c1082e0 2024-09-19T09:31:46.484 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:46.484 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T09:31:46.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 msgr2=0x7fd8480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 0x7fd8480a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0058c0 tx=0x7fd84c00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 msgr2=0x7fd85c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eaa0 tx=0x7fd85400ef60 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 shutdown_connections 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd8480a32d0 0x7fd8480a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85c1082e0 0x7fd85c19ce80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd85c103760 0x7fd85c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 --2- 172.21.15.28:0/3936199204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd85c102db0 0x7fd85c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 >> 172.21.15.28:0/3936199204 conn(0x7fd85c0fc7d0 msgr2=0x7fd85c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 shutdown_connections 2024-09-19T09:31:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:46.484+0000 7fd862836640 1 -- 172.21.15.28:0/3936199204 wait complete. 2024-09-19T09:31:46.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:46 smithi028 ceph-mon[30972]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:46 smithi078 ceph-mon[35619]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:46.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:46 smithi038 ceph-mon[35271]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:31:47.123 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T09:31:47.123 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:31:47.383 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:47.503 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:47 smithi028 ceph-mon[30972]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:31:47.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:47 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3936199204' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:31:47.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:47 smithi078 ceph-mon[35619]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:31:47.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:47 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3936199204' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:31:47.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:47 smithi038 ceph-mon[35271]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:31:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:47 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3936199204' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:31:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 -- 172.21.15.28:0/587268168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 msgr2=0x7ffac41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 --2- 172.21.15.28:0/587268168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41021c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009900 tx=0x7ffac002a410 comp rx=0 tx=0).stop 2024-09-19T09:31:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 -- 172.21.15.28:0/587268168 shutdown_connections 2024-09-19T09:31:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 --2- 172.21.15.28:0/587268168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4102790 0x7ffac41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 --2- 172.21.15.28:0/587268168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4106f10 0x7ffac410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 --2- 172.21.15.28:0/587268168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41021c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 -- 172.21.15.28:0/587268168 >> 172.21.15.28:0/587268168 conn(0x7ffac40fd8c0 msgr2=0x7ffac40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 -- 172.21.15.28:0/587268168 shutdown_connections 2024-09-19T09:31:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.942+0000 7ffacbf56640 1 -- 172.21.15.28:0/587268168 wait complete. 2024-09-19T09:31:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 Processor -- start 2024-09-19T09:31:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 -- start start 2024-09-19T09:31:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4102790 0x7ffac4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4106f10 0x7ffac419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4112010 con 0x7ffac4101de0 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4111e90 con 0x7ffac4106f10 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffacbf56640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4112190 con 0x7ffac4102790 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffac9cf2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffac9cf2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43388/0 (socket says 172.21.15.28:43388) 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffac9cf2640 1 -- 172.21.15.28:0/15654406 learned_addr learned my addr 172.21.15.28:0/15654406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffac94f1640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4102790 0x7ffac4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.943+0000 7ffaca4f3640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4106f10 0x7ffac419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 -- 172.21.15.28:0/15654406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4102790 msgr2=0x7ffac4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4102790 0x7ffac4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 -- 172.21.15.28:0/15654406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4106f10 msgr2=0x7ffac419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4106f10 0x7ffac419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 -- 172.21.15.28:0/15654406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac419d560 con 0x7ffac4101de0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffac9cf2640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005ab0 tx=0x7ffac00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac003f030 con 0x7ffac4101de0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac0003ce0 con 0x7ffac4101de0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac419d7f0 con 0x7ffac4101de0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0035070 con 0x7ffac4101de0 2024-09-19T09:31:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.944+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a5130 con 0x7ffac4101de0 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.945+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac41032c0 con 0x7ffac4101de0 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.946+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ffac0042030 con 0x7ffac4101de0 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.947+0000 7ffabaffd640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 0x7ffab00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.947+0000 7ffac94f1640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 0x7ffab00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.947+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ffac00f04b0 con 0x7ffac4101de0 2024-09-19T09:31:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.948+0000 7ffac94f1640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 0x7ffab00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac40ffde0 tx=0x7ffab4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:47.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:47.950+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffac00bd220 con 0x7ffac4101de0 2024-09-19T09:31:48.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.163+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffac410a0f0 con 0x7ffac4101de0 2024-09-19T09:31:48.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.164+0000 7ffabaffd640 1 -- 172.21.15.28:0/15654406 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7ffac00bcb20 con 0x7ffac4101de0 2024-09-19T09:31:48.166 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:48.166 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:31:48.166 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:31:48.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 msgr2=0x7ffab00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:48.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 0x7ffab00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac40ffde0 tx=0x7ffab4009110 comp rx=0 tx=0).stop 2024-09-19T09:31:48.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 msgr2=0x7ffac41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:48.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005ab0 tx=0x7ffac00042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 shutdown_connections 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffab00a32d0 0x7ffab00a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffac4106f10 0x7ffac419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffac4102790 0x7ffac4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 --2- 172.21.15.28:0/15654406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffac4101de0 0x7ffac41985b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.166+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 >> 172.21.15.28:0/15654406 conn(0x7ffac40fd8c0 msgr2=0x7ffac40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.167+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 shutdown_connections 2024-09-19T09:31:48.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:48.167+0000 7ffacbf56640 1 -- 172.21.15.28:0/15654406 wait complete. 2024-09-19T09:31:48.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:31:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/15654406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:31:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/15654406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:31:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/15654406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:48.872 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:31:49.129 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:31:49.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:49 smithi028 ceph-mon[30972]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-19T09:31:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:31:49.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:49 smithi078 ceph-mon[35619]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:31:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:49 smithi038 ceph-mon[35271]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:31:49.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 -- 172.21.15.28:0/2287865700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 msgr2=0x7f59d0106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/2287865700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0106840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009930 tx=0x7f59bc02a420 comp rx=0 tx=0).stop 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 -- 172.21.15.28:0/2287865700 shutdown_connections 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/2287865700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d0106f50 0x7f59d010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/2287865700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0106840 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/2287865700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d01002f0 0x7f59d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 -- 172.21.15.28:0/2287865700 >> 172.21.15.28:0/2287865700 conn(0x7f59d0077210 msgr2=0x7f59d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 -- 172.21.15.28:0/2287865700 shutdown_connections 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.845+0000 7f59d6ab1640 1 -- 172.21.15.28:0/2287865700 wait complete. 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 Processor -- start 2024-09-19T09:31:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 -- start start 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d01002f0 0x7f59d0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 0x7f59d019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0111fe0 con 0x7f59d0100ca0 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0111e60 con 0x7f59d01002f0 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d6ab1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0112160 con 0x7f59d0106f50 2024-09-19T09:31:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43406/0 (socket says 172.21.15.28:43406) 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59cffff640 1 -- 172.21.15.28:0/1833593054 learned_addr learned my addr 172.21.15.28:0/1833593054 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d484d640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d01002f0 0x7f59d0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.846+0000 7f59d504e640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 0x7f59d019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 -- 172.21.15.28:0/1833593054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 msgr2=0x7f59d019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 0x7f59d019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 -- 172.21.15.28:0/1833593054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d01002f0 msgr2=0x7f59d0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d01002f0 0x7f59d0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 -- 172.21.15.28:0/1833593054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d019d4e0 con 0x7f59d0100ca0 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59d504e640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 0x7f59d019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cffff640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005940 tx=0x7f59bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc03f030 con 0x7f59d0100ca0 2024-09-19T09:31:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d019d770 con 0x7f59d0100ca0 2024-09-19T09:31:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59bc003ce0 con 0x7f59d0100ca0 2024-09-19T09:31:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc035070 con 0x7f59d0100ca0 2024-09-19T09:31:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.847+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d01a50b0 con 0x7f59d0100ca0 2024-09-19T09:31:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.848+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d01033b0 con 0x7f59d0100ca0 2024-09-19T09:31:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.849+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f59bc042030 con 0x7f59d0100ca0 2024-09-19T09:31:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.850+0000 7f59cdffb640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 0x7f59c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.850+0000 7f59d484d640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 0x7f59c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.850+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f59bc0f0230 con 0x7f59d0100ca0 2024-09-19T09:31:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.851+0000 7f59d484d640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 0x7f59c00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009990 tx=0x7f59c4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:49.853+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59bc0bce20 con 0x7f59d0100ca0 2024-09-19T09:31:50.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.065+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59d010a120 con 0x7f59d0100ca0 2024-09-19T09:31:50.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.065+0000 7f59cdffb640 1 -- 172.21.15.28:0/1833593054 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7f59bc0bc720 con 0x7f59d0100ca0 2024-09-19T09:31:50.067 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:50.068 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:31:50.068 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:31:50.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 msgr2=0x7f59c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:50.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 0x7f59c00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009990 tx=0x7f59c4009110 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 msgr2=0x7f59d0198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005940 tx=0x7f59bc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 shutdown_connections 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59c00a32d0 0x7f59c00a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59d0106f50 0x7f59d019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59d0100ca0 0x7f59d0198a70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 --2- 172.21.15.28:0/1833593054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59d01002f0 0x7f59d0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 >> 172.21.15.28:0/1833593054 conn(0x7f59d0077210 msgr2=0x7f59d01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 shutdown_connections 2024-09-19T09:31:50.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:50.068+0000 7f59d6ab1640 1 -- 172.21.15.28:0/1833593054 wait complete. 2024-09-19T09:31:50.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:50 smithi038 ceph-mon[35271]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:31:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:50 smithi038 ceph-mon[35271]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-19T09:31:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:50 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1833593054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:50.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:50 smithi028 ceph-mon[30972]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:31:50.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:50 smithi028 ceph-mon[30972]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-19T09:31:50.684 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:50 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1833593054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:50.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:31:50.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:50 smithi078 ceph-mon[35619]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:31:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:50 smithi078 ceph-mon[35619]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-19T09:31:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:50 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1833593054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:50.969 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:51.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 -- 172.21.15.28:0/3961506632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c102ef0 msgr2=0x7fc42c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:51.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 --2- 172.21.15.28:0/3961506632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c102ef0 0x7fc42c10da40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc424009ca0 tx=0x7fc42402a950 comp rx=0 tx=0).stop 2024-09-19T09:31:51.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 -- 172.21.15.28:0/3961506632 shutdown_connections 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 --2- 172.21.15.28:0/3961506632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c102ef0 0x7fc42c10da40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 --2- 172.21.15.28:0/3961506632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102570 0x7fc42c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 --2- 172.21.15.28:0/3961506632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c100e20 0x7fc42c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 -- 172.21.15.28:0/3961506632 >> 172.21.15.28:0/3961506632 conn(0x7fc42c0fc7d0 msgr2=0x7fc42c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 -- 172.21.15.28:0/3961506632 shutdown_connections 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.395+0000 7fc432646640 1 -- 172.21.15.28:0/3961506632 wait complete. 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 Processor -- start 2024-09-19T09:31:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 -- start start 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c102570 0x7fc42c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102ef0 0x7fc42c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c110fa0 con 0x7fc42c100e20 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c110e20 con 0x7fc42c102570 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc432646640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c111120 con 0x7fc42c102ef0 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43414/0 (socket says 172.21.15.28:43414) 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 -- 172.21.15.28:0/4210003769 learned_addr learned my addr 172.21.15.28:0/4210003769 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42b7fe640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c102570 0x7fc42c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:51.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc430be3640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102ef0 0x7fc42c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 -- 172.21.15.28:0/4210003769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102ef0 msgr2=0x7fc42c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102ef0 0x7fc42c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 -- 172.21.15.28:0/4210003769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c102570 msgr2=0x7fc42c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c102570 0x7fc42c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.396+0000 7fc42bfff640 1 -- 172.21.15.28:0/4210003769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a6a10 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc42bfff640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d950 tx=0x7fc41c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a6d00 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c00de80 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00f620 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.397+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a71e0 con 0x7fc42c100e20 2024-09-19T09:31:51.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.398+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c101200 con 0x7fc42c100e20 2024-09-19T09:31:51.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.399+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc41c009c30 con 0x7fc42c100e20 2024-09-19T09:31:51.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.400+0000 7fc4297fa640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 0x7fc4140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:51.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.400+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc41c0cf6e0 con 0x7fc42c100e20 2024-09-19T09:31:51.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.400+0000 7fc42b7fe640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 0x7fc4140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:51.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.401+0000 7fc42b7fe640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 0x7fc4140a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0fe710 tx=0x7fc420009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:51.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.405+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc41c09c2d0 con 0x7fc42c100e20 2024-09-19T09:31:51.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.581+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc42c10a0a0 con 0x7fc42c100e20 2024-09-19T09:31:51.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.582+0000 7fc4297fa640 1 -- 172.21.15.28:0/4210003769 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22580 (secure 0 0 0) 0x7fc41c09bbd0 con 0x7fc42c100e20 2024-09-19T09:31:51.584 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:51.584 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:31:49.348824+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 msgr2=0x7fc4140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 0x7fc4140a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0fe710 tx=0x7fc420009110 comp rx=0 tx=0).stop 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 msgr2=0x7fc42c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d950 tx=0x7fc41c00dd50 comp rx=0 tx=0).stop 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 shutdown_connections 2024-09-19T09:31:51.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4140a32d0 0x7fc4140a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc42c102570 0x7fc42c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c102ef0 0x7fc42c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 --2- 172.21.15.28:0/4210003769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c100e20 0x7fc42c0fcc80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 >> 172.21.15.28:0/4210003769 conn(0x7fc42c0fc7d0 msgr2=0x7fc42c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 shutdown_connections 2024-09-19T09:31:51.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:51.584+0000 7fc432646640 1 -- 172.21.15.28:0/4210003769 wait complete. 2024-09-19T09:31:51.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:51 smithi038 ceph-mon[35271]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-19T09:31:51.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:51 smithi028 ceph-mon[30972]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-19T09:31:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:51 smithi078 ceph-mon[35619]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-19T09:31:52.130 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:31:52.381 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:52 smithi028 ceph-mon[30972]: pgmap v314: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:31:52.381 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:52 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4210003769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:52.401 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:52.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:52 smithi038 ceph-mon[35271]: pgmap v314: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:31:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:52 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4210003769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:52 smithi078 ceph-mon[35619]: pgmap v314: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:31:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:52 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4210003769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:31:52.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 -- 172.21.15.28:0/1039380208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 msgr2=0x7f1790106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:52.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 --2- 172.21.15.28:0/1039380208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790106f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1780009ca0 tx=0x7f178002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 -- 172.21.15.28:0/1039380208 shutdown_connections 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 --2- 172.21.15.28:0/1039380208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 0x7f179010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 --2- 172.21.15.28:0/1039380208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 0x7f1790100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 --2- 172.21.15.28:0/1039380208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790106f80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 -- 172.21.15.28:0/1039380208 >> 172.21.15.28:0/1039380208 conn(0x7f1790077210 msgr2=0x7f17900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 -- 172.21.15.28:0/1039380208 shutdown_connections 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 -- 172.21.15.28:0/1039380208 wait complete. 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.860+0000 7f17955f5640 1 Processor -- start 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 -- start start 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 0x7f17901012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 0x7f17901017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:52.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112090 con 0x7f1790106ba0 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111f10 con 0x7f1790100a70 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f17955f5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112210 con 0x7f17901000d0 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43440/0 (socket says 172.21.15.28:43440) 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f178f7fe640 1 -- 172.21.15.28:0/415270364 learned_addr learned my addr 172.21.15.28:0/415270364 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f178effd640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 0x7f17901012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.861+0000 7f178e7fc640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 0x7f17901017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 -- 172.21.15.28:0/415270364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 msgr2=0x7f17901012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 0x7f17901012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 -- 172.21.15.28:0/415270364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 msgr2=0x7f17901017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 0x7f17901017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 -- 172.21.15.28:0/415270364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a6990 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f178f7fe640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f178800c970 tx=0x7f178800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788007bb0 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a6c80 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788007d50 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a71c0 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.862+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788018800 con 0x7f1790106ba0 2024-09-19T09:31:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.863+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c005290 con 0x7f1790106ba0 2024-09-19T09:31:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.864+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f178800f450 con 0x7f1790106ba0 2024-09-19T09:31:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.865+0000 7f1777fff640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 0x7f17780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.865+0000 7f178effd640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 0x7f17780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.865+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f17880cf810 con 0x7f1790106ba0 2024-09-19T09:31:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.866+0000 7f178effd640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 0x7f17780a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1780009a60 tx=0x7f1780031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:52.869+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1788014030 con 0x7f1790106ba0 2024-09-19T09:31:53.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.082+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f177c005850 con 0x7f1790106ba0 2024-09-19T09:31:53.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.083+0000 7f1777fff640 1 -- 172.21.15.28:0/415270364 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7f178809c400 con 0x7f1790106ba0 2024-09-19T09:31:53.085 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:31:53.085 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:31:53.085 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 msgr2=0x7f17780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 0x7f17780a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1780009a60 tx=0x7f1780031000 comp rx=0 tx=0).stop 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 msgr2=0x7f1790101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f178800c970 tx=0x7f178800ce30 comp rx=0 tx=0).stop 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 shutdown_connections 2024-09-19T09:31:53.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f17780a3350 0x7f17780a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1790100a70 0x7f17901017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1790106ba0 0x7f1790101d30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 --2- 172.21.15.28:0/415270364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901000d0 0x7f17901012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 >> 172.21.15.28:0/415270364 conn(0x7f1790077210 msgr2=0x7f1790104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 shutdown_connections 2024-09-19T09:31:53.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:53.085+0000 7f17955f5640 1 -- 172.21.15.28:0/415270364 wait complete. 2024-09-19T09:31:53.673 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:53 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/415270364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:53.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:53 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/415270364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:53.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:30:43.054690+0000', 'modified': '2024-09-19T09:30:54.185943+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24499, 'mds_1': 24482, 'mds_2': 24478, 'mds_3': 24472, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6839/1981224443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6838', 'nonce': 1981224443}, {'type': 'v1', 'addr': '172.21.15.28:6839', 'nonce': 1981224443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6841/438294996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6840', 'nonce': 438294996}, {'type': 'v1', 'addr': '172.21.15.28:6841', 'nonce': 438294996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3013823435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3013823435}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3013823435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/347872214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 347872214}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 347872214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6833/2992347158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 2992347158}, {'type': 'v1', 'addr': '172.21.15.78:6833', 'nonce': 2992347158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.38:6837/560457008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 560457008}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 560457008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.78:6835/2086564778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 2086564778}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 2086564778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.78:6837/2648819818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2648819818}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2648819818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.38:6839/3114720816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 3114720816}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 3114720816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.78:6839/3581897338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3581897338}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3581897338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24499, 24482, 24478, 24472, 24455]} 2024-09-19T09:31:53.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:31:53.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:31:53.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:53 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/415270364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:31:53.958 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:54.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 -- 172.21.15.28:0/1509503525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff444102d30 msgr2=0x7ff444103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 --2- 172.21.15.28:0/1509503525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff444102d30 0x7ff444103110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff434009930 tx=0x7ff43402a420 comp rx=0 tx=0).stop 2024-09-19T09:31:54.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 -- 172.21.15.28:0/1509503525 shutdown_connections 2024-09-19T09:31:54.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 --2- 172.21.15.28:0/1509503525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441082a0 0x7ff44410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 --2- 172.21.15.28:0/1509503525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4441036e0 0x7ff444107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 --2- 172.21.15.28:0/1509503525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff444102d30 0x7ff444103110 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 -- 172.21.15.28:0/1509503525 >> 172.21.15.28:0/1509503525 conn(0x7ff4440fc7d0 msgr2=0x7ff4440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 -- 172.21.15.28:0/1509503525 shutdown_connections 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.392+0000 7ff44888b640 1 -- 172.21.15.28:0/1509503525 wait complete. 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 Processor -- start 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 -- start start 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441036e0 0x7ff4441a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff4441120a0 con 0x7ff4441082a0 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff444111f20 con 0x7ff4441036e0 2024-09-19T09:31:54.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44888b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff444112220 con 0x7ff444102d30 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff442d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36276/0 (socket says 172.21.15.28:36276) 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff442d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45022/0 (socket says 172.21.15.28:45022) 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 -- 172.21.15.28:0/1827407120 learned_addr learned my addr 172.21.15.28:0/1827407120 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff43bfff640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441036e0 0x7ff4441a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 -- 172.21.15.28:0/1827407120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441036e0 msgr2=0x7ff4441a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441036e0 0x7ff4441a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 -- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 msgr2=0x7ff44419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff44259c640 1 -- 172.21.15.28:0/1827407120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44419ad40 con 0x7ff444102d30 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.393+0000 7ff442d9d640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:54.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff44259c640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff434009230 tx=0x7ff434004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff43b7fe640 1 -- 172.21.15.28:0/1827407120 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43403f030 con 0x7ff444102d30 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44419afd0 con 0x7ff444102d30 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff43b7fe640 1 -- 172.21.15.28:0/1827407120 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff434003ad0 con 0x7ff444102d30 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff43b7fe640 1 -- 172.21.15.28:0/1827407120 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434035070 con 0x7ff444102d30 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.394+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4441a7320 con 0x7ff444102d30 2024-09-19T09:31:54.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.396+0000 7ff43b7fe640 1 -- 172.21.15.28:0/1827407120 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff434042030 con 0x7ff444102d30 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 Processor -- start 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 -- start start 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a319d0 0x55c371a31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 0x55c371a3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c371998090 con 0x55c371a3cbb0 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c37193fa70 con 0x55c371a319d0 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44ac64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c371994050 con 0x55c371a322f0 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff44259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 0x55c371a3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45024/0 (socket says 172.21.15.28:45024) 2024-09-19T09:31:54.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff43bfff640 1 -- 172.21.15.28:0/3770217714 learned_addr learned my addr 172.21.15.28:0/3770217714 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff43b7fe640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 0x7ff4300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff442d9d640 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a319d0 0x55c371a31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.397+0000 7ff442d9d640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 0x7ff4300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 -- 172.21.15.28:0/3770217714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 msgr2=0x55c371a3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 0x55c371a3c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 -- 172.21.15.28:0/3770217714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a319d0 msgr2=0x55c371a31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a319d0 0x55c371a31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 -- 172.21.15.28:0/3770217714 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37194c130 con 0x55c371a3cbb0 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43b7fe640 1 -- 172.21.15.28:0/1827407120 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff4340eff30 con 0x7ff444102d30 2024-09-19T09:31:54.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff44259c640 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 0x55c371a3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff442d9d640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 0x7ff4300a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff43c007fd0 tx=0x7ff43c007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00bbf0 tx=0x7ff42c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e9359dc9dd2b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff438ff9640 1 -- 172.21.15.28:0/3770217714 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004030 con 0x55c371a3cbb0 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff438ff9640 1 -- 172.21.15.28:0/3770217714 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c0041d0 con 0x55c371a3cbb0 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.398+0000 7ff438ff9640 1 -- 172.21.15.28:0/3770217714 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c011680 con 0x55c371a3cbb0 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- 172.21.15.28:0/3770217714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 msgr2=0x55c371a3efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00bbf0 tx=0x7ff42c0107a0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- 172.21.15.28:0/3770217714 shutdown_connections 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a3cbb0 0x55c371a3efa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a322f0 0x55c371a3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3770217714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a319d0 0x55c371a31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- 172.21.15.28:0/3770217714 >> 172.21.15.28:0/3770217714 conn(0x55c371a2df60 msgr2=0x55c371a2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- 172.21.15.28:0/3770217714 shutdown_connections 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- 172.21.15.28:0/3770217714 wait complete. 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.399+0000 7ff44ac64740 1 -- start start 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 Processor -- start 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a39750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c371a3f780 con 0x55c371a319d0 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c371952f20 con 0x55c371a322f0 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c371a3f900 con 0x55c371a3cbb0 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44ac64740 10 client.0 _ll_register_callbacks cb 0x55c37191c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45034/0 (socket says 172.21.15.28:45034) 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 -- 172.21.15.28:0/3242636639 learned_addr learned my addr 172.21.15.28:0/3242636639 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a39750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff442d9d640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 -- 172.21.15.28:0/3242636639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 msgr2=0x55c371a39c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a39c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 -- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 msgr2=0x55c371a39750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a39750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 -- 172.21.15.28:0/3242636639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3717092b0 con 0x55c371a319d0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff44259c640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff434009900 tx=0x7ff4340b6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8909e5c52d72920e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.400+0000 7ff429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434049070 con 0x55c371a319d0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff4340bcb70 con 0x55c371a319d0 2024-09-19T09:31:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c0041d0 con 0x55c371a319d0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4340bc470 con 0x55c371a319d0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 10 client.14988 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c371a4b050 con 0x55c371a319d0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c0041d0 con 0x55c371a319d0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff43403bc30 con 0x55c371a319d0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 10 client.14988 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 10 client.14988 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 20 client.14988 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371a4d660 0x55c371a4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 4 client.14988 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.401+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3719a5600 con 0x55c371a4d660 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.402+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371a4d660 0x55c371a4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.402+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371a4d660 0x55c371a4ead0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68347ad57a087559 server_cookie=3b2363e03457805b in_seq=0 out_seq=0 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.402+0000 7ff429ffb640 10 client.14988 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff429ffb640 1 -- 172.21.15.28:0/3242636639 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3719a5600 con 0x55c371a4d660 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff429ffb640 10 client.14988 handle_command_reply: tid=0 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 client.14988 shutdown 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 msgr2=0x55c371a3c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff434009900 tx=0x7ff4340b6ed0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 shutdown_connections 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371a4d660 0x55c371a4ead0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a39750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3242636639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a3c400 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 >> 172.21.15.28:0/3242636639 conn(0x55c371a2df60 msgr2=0x55c371a33c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff429ffb640 0 client.14988 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:31:54.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 shutdown_connections 2024-09-19T09:31:54.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 1 -- 172.21.15.28:0/3242636639 wait complete. 2024-09-19T09:31:54.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.403+0000 7ff44ac64740 20 client.14988 trim_cache size 0 max 16384 2024-09-19T09:31:54.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 Processor -- start 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 -- start start 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4ce10 con 0x55c371a319d0 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4d4e0 con 0x55c371a322f0 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44ac64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4dbb0 con 0x55c371a3cbb0 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45048/0 (socket says 172.21.15.28:45048) 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.421+0000 7ff44259c640 1 -- 172.21.15.28:0/2279244879 learned_addr learned my addr 172.21.15.28:0/2279244879 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:54.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff43bfff640 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff442d9d640 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 -- 172.21.15.28:0/2279244879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 msgr2=0x55c371a2e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a2e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 -- 172.21.15.28:0/2279244879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 msgr2=0x55c371a4e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a4e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 -- 172.21.15.28:0/2279244879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c371a32b70 con 0x55c371a319d0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff44259c640 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff4340dc010 tx=0x7ff4340ba980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1cc29277e4fa2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff42bfff640 1 -- 172.21.15.28:0/2279244879 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340d9070 con 0x55c371a319d0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff42bfff640 1 -- 172.21.15.28:0/2279244879 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4340c5030 con 0x55c371a319d0 2024-09-19T09:31:54.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.422+0000 7ff42bfff640 1 -- 172.21.15.28:0/2279244879 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434049070 con 0x55c371a319d0 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- 172.21.15.28:0/2279244879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 msgr2=0x55c371a4cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff4340dc010 tx=0x7ff4340ba980 comp rx=0 tx=0).stop 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- 172.21.15.28:0/2279244879 shutdown_connections 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a4e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- 172.21.15.28:0/2279244879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a4cf90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- 172.21.15.28:0/2279244879 >> 172.21.15.28:0/2279244879 conn(0x55c371a2df60 msgr2=0x55c371a39710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- 172.21.15.28:0/2279244879 shutdown_connections 2024-09-19T09:31:54.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- 172.21.15.28:0/2279244879 wait complete. 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- start start 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 Processor -- start 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a51890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4e950 con 0x55c371a319d0 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4e280 con 0x55c371a322f0 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c371a4f020 con 0x55c371a3cbb0 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44ac64740 10 client.0 _ll_register_callbacks cb 0x7ff434009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.423+0000 7ff44259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45050/0 (socket says 172.21.15.28:45050) 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 -- 172.21.15.28:0/3488678652 learned_addr learned my addr 172.21.15.28:0/3488678652 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:54.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a51890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff442d9d640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 -- 172.21.15.28:0/3488678652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 msgr2=0x55c371a51ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a51ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 -- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 msgr2=0x55c371a51890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a51890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 -- 172.21.15.28:0/3488678652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c371a526d0 con 0x55c371a319d0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44259c640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4340bc400 tx=0x7ff434005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57706639cace9b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434049070 con 0x55c371a319d0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff4340c4da0 con 0x55c371a319d0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4340c5030 con 0x55c371a319d0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4340d9050 con 0x55c371a319d0 2024-09-19T09:31:54.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44ac64740 10 client.14994 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.424+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c371a526d0 con 0x55c371a319d0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4340c5030 con 0x55c371a319d0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff4340b81c0 con 0x55c371a319d0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 10 client.14994 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 10 client.14994 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 20 client.14994 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371abb2c0 0x55c371ab9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 4 client.14994 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "status"}) -- 0x55c37199fb10 con 0x55c371abb2c0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.425+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371abb2c0 0x55c371ab9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff43bfff640 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371abb2c0 0x55c371ab9fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=365932de5ad14add server_cookie=4afd13812e23f9ce in_seq=0 out_seq=0 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff429ffb640 10 client.14994 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:31:54.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff429ffb640 1 -- 172.21.15.28:0/3488678652 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c37199fb10 con 0x55c371abb2c0 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff429ffb640 10 client.14994 handle_command_reply: tid=0 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 client.14994 shutdown 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 msgr2=0x55c371a57570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4340bc400 tx=0x7ff434005e50 comp rx=0 tx=0).stop 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 shutdown_connections 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c371abb2c0 0x55c371ab9fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c371a322f0 0x55c371a51890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c371a3cbb0 0x55c371a51ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 --2- 172.21.15.28:0/3488678652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c371a319d0 0x55c371a57570 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 >> 172.21.15.28:0/3488678652 conn(0x55c371a2df60 msgr2=0x55c371a2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff429ffb640 0 client.14994 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 shutdown_connections 2024-09-19T09:31:54.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 1 -- 172.21.15.28:0/3488678652 wait complete. 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.426+0000 7ff44ac64740 20 client.14994 trim_cache size 0 max 16384 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 4, 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24455, 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 61.248634971999998, 2024-09-19T09:31:54.434 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 92.139429781999993 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 msgr2=0x7ff4300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 0x7ff4300a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff43c007fd0 tx=0x7ff43c007520 comp rx=0 tx=0).stop 2024-09-19T09:31:54.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 msgr2=0x7ff4441a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff434009230 tx=0x7ff434004270 comp rx=0 tx=0).stop 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 shutdown_connections 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff4300a32d0 0x7ff4300a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4441036e0 0x7ff4441a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4441082a0 0x7ff44419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 --2- 172.21.15.28:0/1827407120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff444102d30 0x7ff4441a0450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 >> 172.21.15.28:0/1827407120 conn(0x7ff4440fc7d0 msgr2=0x7ff4440fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 shutdown_connections 2024-09-19T09:31:54.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:54.427+0000 7ff44888b640 1 -- 172.21.15.28:0/1827407120 wait complete. 2024-09-19T09:31:54.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:54 smithi038 ceph-mon[35271]: pgmap v315: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-19T09:31:54.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:54 smithi028 ceph-mon[30972]: pgmap v315: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-19T09:31:54.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:54 smithi078 ceph-mon[35619]: pgmap v315: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-19T09:31:55.142 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:31:55.400 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:55.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:55 smithi028 ceph-mon[30972]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-09-19T09:31:55.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:55 smithi028 ceph-mon[30972]: Cluster is now healthy 2024-09-19T09:31:55.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 -- 172.21.15.28:0/3905784492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00106a80 msgr2=0x7faa00106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 --2- 172.21.15.28:0/3905784492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00106a80 0x7faa00106e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009930 tx=0x7fa9f002a420 comp rx=0 tx=0).stop 2024-09-19T09:31:55.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 -- 172.21.15.28:0/3905784492 shutdown_connections 2024-09-19T09:31:55.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 --2- 172.21.15.28:0/3905784492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101420 0x7faa0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 --2- 172.21.15.28:0/3905784492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00100a80 0x7faa00100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 --2- 172.21.15.28:0/3905784492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00106a80 0x7faa00106e60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 -- 172.21.15.28:0/3905784492 >> 172.21.15.28:0/3905784492 conn(0x7faa000fc7a0 msgr2=0x7faa000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 -- 172.21.15.28:0/3905784492 shutdown_connections 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.826+0000 7faa064af640 1 -- 172.21.15.28:0/3905784492 wait complete. 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 Processor -- start 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 -- start start 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00100a80 0x7faa000fce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00106a80 0x7faa000fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110e80 con 0x7faa00101420 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110d00 con 0x7faa00100a80 2024-09-19T09:31:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa064af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa00111000 con 0x7faa00106a80 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45064/0 (socket says 172.21.15.28:45064) 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2604440094 learned_addr learned my addr 172.21.15.28:0/2604440094 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7fa9fffff640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00100a80 0x7faa000fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.827+0000 7faa04a4c640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00106a80 0x7faa000fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2604440094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00106a80 msgr2=0x7faa000fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00106a80 0x7faa000fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2604440094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00100a80 msgr2=0x7faa000fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00100a80 0x7faa000fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2604440094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001a6980 con 0x7faa00101420 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c970 tx=0x7fa9f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9fd7fa640 1 -- 172.21.15.28:0/2604440094 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4007bb0 con 0x7faa00101420 2024-09-19T09:31:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9fd7fa640 1 -- 172.21.15.28:0/2604440094 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f4007d50 con 0x7faa00101420 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a6c70 con 0x7faa00101420 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7fa9fd7fa640 1 -- 172.21.15.28:0/2604440094 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400f450 con 0x7faa00101420 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.828+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a7160 con 0x7faa00101420 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.830+0000 7fa9fd7fa640 1 -- 172.21.15.28:0/2604440094 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa9f4005020 con 0x7faa00101420 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.830+0000 7faa08888740 1 Processor -- start 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.830+0000 7faa08888740 1 -- start start 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562785cff090 con 0x562785d989d0 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562785ca6a70 con 0x562785da3bb0 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa08888740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562785cfb050 con 0x562785d992f0 2024-09-19T09:31:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45080/0 (socket says 172.21.15.28:45080) 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 -- 172.21.15.28:0/4056534373 learned_addr learned my addr 172.21.15.28:0/4056534373 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7fa9fffff640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7fa9fd7fa640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 0x7fa9e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 -- 172.21.15.28:0/4056534373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 msgr2=0x562785da3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 -- 172.21.15.28:0/4056534373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 msgr2=0x562785da5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 -- 172.21.15.28:0/4056534373 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562785cb3130 con 0x562785d989d0 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7fa9fffff640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7fa9fd7fa640 1 -- 172.21.15.28:0/2604440094 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f40cf910 con 0x7faa00101420 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 0x7fa9e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.831+0000 7faa04a4c640 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009eb0 tx=0x7fa9f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3072355aa037c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7fa9e67fc640 1 -- 172.21.15.28:0/4056534373 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f801a610 con 0x562785d989d0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7fa9e67fc640 1 -- 172.21.15.28:0/4056534373 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f8012e40 con 0x562785d989d0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7fa9e67fc640 1 -- 172.21.15.28:0/4056534373 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80108e0 con 0x562785d989d0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa04a4c640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 0x7fa9e80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8020000 tx=0x7fa9f801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 -- 172.21.15.28:0/4056534373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 msgr2=0x562785d98db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009eb0 tx=0x7fa9f800f730 comp rx=0 tx=0).stop 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 -- 172.21.15.28:0/4056534373 shutdown_connections 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 --2- 172.21.15.28:0/4056534373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785d98db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.832+0000 7faa08888740 1 -- 172.21.15.28:0/4056534373 >> 172.21.15.28:0/4056534373 conn(0x562785d94f60 msgr2=0x562785d95340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- 172.21.15.28:0/4056534373 shutdown_connections 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- 172.21.15.28:0/4056534373 wait complete. 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- start start 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 Processor -- start 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562785da6780 con 0x562785d989d0 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562785cb9f20 con 0x562785da3bb0 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562785da6900 con 0x562785d992f0 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7faa08888740 10 client.0 _ll_register_callbacks cb 0x562785c83030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.833+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45092/0 (socket says 172.21.15.28:45092) 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 -- 172.21.15.28:0/3519889198 learned_addr learned my addr 172.21.15.28:0/3519889198 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7faa04a4c640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 -- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 msgr2=0x562785da0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 -- 172.21.15.28:0/3519889198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 msgr2=0x562785da0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 -- 172.21.15.28:0/3519889198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562785a702b0 con 0x562785d989d0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9fffff640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0031550 tx=0x7fa9f00318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a70b197ef9f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0035070 con 0x562785d989d0 2024-09-19T09:31:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f0004240 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562785d0c600 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0004440 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7faa08888740 10 client.15006 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.834+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562785db2090 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562785d0c600 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f0004ba0 con 0x562785d989d0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 10 client.15006 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 10 client.15006 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 20 client.15006 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785db46a0 0x562785db5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 4 client.15006 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562785d06b10 con 0x562785db46a0 2024-09-19T09:31:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.835+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785db46a0 0x562785db5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.836+0000 7fa9df7fe640 10 client.15006 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.836+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785db46a0 0x562785db5b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca64418b4e887345 server_cookie=c0daa4dba1935042 in_seq=0 out_seq=0 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.836+0000 7fa9df7fe640 1 -- 172.21.15.28:0/3519889198 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562785d06b10 con 0x562785db46a0 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.836+0000 7fa9df7fe640 10 client.15006 handle_command_reply: tid=0 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.836+0000 7faa08888740 1 client.15006 shutdown 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 msgr2=0x562785da3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0031550 tx=0x7fa9f00318d0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 shutdown_connections 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785db46a0 0x562785db5b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785da0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d992f0 0x562785da0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 --2- 172.21.15.28:0/3519889198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d989d0 0x562785da3430 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 >> 172.21.15.28:0/3519889198 conn(0x562785d94f60 msgr2=0x562785d9ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7fa9df7fe640 0 client.15006 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:31:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 shutdown_connections 2024-09-19T09:31:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 1 -- 172.21.15.28:0/3519889198 wait complete. 2024-09-19T09:31:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.837+0000 7faa08888740 20 client.15006 trim_cache size 0 max 16384 2024-09-19T09:31:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 Processor -- start 2024-09-19T09:31:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 -- start start 2024-09-19T09:31:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 0x562785db4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785da3bb0 0x562785dbe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562785db3e50 con 0x562785d992f0 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562785db4520 con 0x562785d989d0 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7faa08888740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562785db4bf0 con 0x562785da3bb0 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45100/0 (socket says 172.21.15.28:45100) 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2076901438 learned_addr learned my addr 172.21.15.28:0/2076901438 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9fffff640 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 0x562785db4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2076901438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785da3bb0 msgr2=0x562785dbe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785da3bb0 0x562785dbe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2076901438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 msgr2=0x562785db4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 0x562785db4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/2076901438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562785da3050 con 0x562785d992f0 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9fffff640 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 0x562785db4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.855+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9f409bd90 tx=0x7fa9f40d1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d89d725532e6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7fa9e5ffb640 1 -- 172.21.15.28:0/2076901438 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f401d040 con 0x562785d992f0 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7fa9e5ffb640 1 -- 172.21.15.28:0/2076901438 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f409c500 con 0x562785d992f0 2024-09-19T09:31:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7fa9e5ffb640 1 -- 172.21.15.28:0/2076901438 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4099f30 con 0x562785d992f0 2024-09-19T09:31:55.859 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:55 smithi078 ceph-mon[35619]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-09-19T09:31:55.859 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:55 smithi078 ceph-mon[35619]: Cluster is now healthy 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 -- 172.21.15.28:0/2076901438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 msgr2=0x562785dbddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9f409bd90 tx=0x7fa9f40d1e00 comp rx=0 tx=0).stop 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 -- 172.21.15.28:0/2076901438 shutdown_connections 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785da3bb0 0x562785dbe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbddf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 --2- 172.21.15.28:0/2076901438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785d989d0 0x562785db4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 -- 172.21.15.28:0/2076901438 >> 172.21.15.28:0/2076901438 conn(0x562785d94f60 msgr2=0x562785d9f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 -- 172.21.15.28:0/2076901438 shutdown_connections 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.856+0000 7faa08888740 1 -- 172.21.15.28:0/2076901438 wait complete. 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 -- start start 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 Processor -- start 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d989d0 0x562785dbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785e1cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562785db5990 con 0x562785d992f0 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562785db52c0 con 0x562785da3bb0 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562785db6060 con 0x562785d989d0 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa08888740 10 client.0 _ll_register_callbacks cb 0x7fa9f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45106/0 (socket says 172.21.15.28:45106) 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/1489679116 learned_addr learned my addr 172.21.15.28:0/1489679116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.857+0000 7faa04a4c640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785e1cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9fffff640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d989d0 0x562785dbca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d989d0 msgr2=0x562785dbca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d989d0 0x562785dbca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/1489679116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 msgr2=0x562785e1cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785e1cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 -- 172.21.15.28:0/1489679116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562785e1d2e0 con 0x562785d992f0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7faa04a4c640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785e1cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9ff7fe640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40cd370 tx=0x7fa9f409bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd87a71c33bdb9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f401d070 con 0x562785d992f0 2024-09-19T09:31:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f40cd540 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f409c500 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa9f406f180 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7faa08888740 10 client.15012 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562785e1e480 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.858+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f409c500 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f4066070 con 0x562785d992f0 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 10 client.15012 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 10 client.15012 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 20 client.15012 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785e22a70 0x562785e21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 4 client.15012 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "status"}) -- 0x562785c830e0 con 0x562785e22a70 2024-09-19T09:31:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7fa9fffff640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785e22a70 0x562785e21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7fa9fffff640 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785e22a70 0x562785e21780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72460e4e61940ba7 server_cookie=3e50d0f8de4c9632 in_seq=0 out_seq=0 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.859+0000 7fa9df7fe640 10 client.15012 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7fa9df7fe640 1 -- 172.21.15.28:0/1489679116 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562785c830e0 con 0x562785e22a70 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7fa9df7fe640 10 client.15012 handle_command_reply: tid=0 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 client.15012 shutdown 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 msgr2=0x562785dbcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40cd370 tx=0x7fa9f409bef0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 shutdown_connections 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x562785e22a70 0x562785e21780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562785da3bb0 0x562785e1cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562785d992f0 0x562785dbcfb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 --2- 172.21.15.28:0/1489679116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562785d989d0 0x562785dbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 >> 172.21.15.28:0/1489679116 conn(0x562785d94f60 msgr2=0x562785d9a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7fa9df7fe640 0 client.15012 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 shutdown_connections 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 1 -- 172.21.15.28:0/1489679116 wait complete. 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.860+0000 7faa08888740 20 client.15012 trim_cache size 0 max 16384 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 3, 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24472, 2024-09-19T09:31:55.867 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 63.686493794, 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 98.520200552000006 2024-09-19T09:31:55.868 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 msgr2=0x7fa9e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 0x7fa9e80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8020000 tx=0x7fa9f801d000 comp rx=0 tx=0).stop 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 msgr2=0x7faa000fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c970 tx=0x7fa9f400ce30 comp rx=0 tx=0).stop 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 shutdown_connections 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32d0 0x7fa9e80a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa00106a80 0x7faa000fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa00101420 0x7faa000fd3c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 --2- 172.21.15.28:0/2604440094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00100a80 0x7faa000fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 >> 172.21.15.28:0/2604440094 conn(0x7faa000fc7a0 msgr2=0x7faa0006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 shutdown_connections 2024-09-19T09:31:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:55.861+0000 7faa064af640 1 -- 172.21.15.28:0/2604440094 wait complete. 2024-09-19T09:31:55.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:55 smithi038 ceph-mon[35271]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-09-19T09:31:55.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:55 smithi038 ceph-mon[35271]: Cluster is now healthy 2024-09-19T09:31:56.468 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:31:56.722 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:56.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:56 smithi028 ceph-mon[30972]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 28 B/s, 0 objects/s recovering 2024-09-19T09:31:56.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:56 smithi078 ceph-mon[35619]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 28 B/s, 0 objects/s recovering 2024-09-19T09:31:56.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:56 smithi038 ceph-mon[35271]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 28 B/s, 0 objects/s recovering 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.156+0000 7fd709f03640 1 -- 172.21.15.28:0/2770688064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 msgr2=0x7fd704106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.156+0000 7fd709f03640 1 --2- 172.21.15.28:0/2770688064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd704106e60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009930 tx=0x7fd6f402a420 comp rx=0 tx=0).stop 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 -- 172.21.15.28:0/2770688064 shutdown_connections 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 --2- 172.21.15.28:0/2770688064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 0x7fd704100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 --2- 172.21.15.28:0/2770688064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 0x7fd70410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 --2- 172.21.15.28:0/2770688064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd704106e60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 -- 172.21.15.28:0/2770688064 >> 172.21.15.28:0/2770688064 conn(0x7fd7040fc7a0 msgr2=0x7fd7040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 -- 172.21.15.28:0/2770688064 shutdown_connections 2024-09-19T09:31:57.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 -- 172.21.15.28:0/2770688064 wait complete. 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 Processor -- start 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.157+0000 7fd709f03640 1 -- start start 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 0x7fd704199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 0x7fd704199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd704110ea0 con 0x7fd704106a80 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd704110d20 con 0x7fd704100a80 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd709f03640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd704111020 con 0x7fd704101420 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45120/0 (socket says 172.21.15.28:45120) 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 -- 172.21.15.28:0/2600533316 learned_addr learned my addr 172.21.15.28:0/2600533316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:57.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd7037fe640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 0x7fd704199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd702ffd640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 0x7fd704199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 -- 172.21.15.28:0/2600533316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 msgr2=0x7fd704199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 0x7fd704199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 -- 172.21.15.28:0/2600533316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 msgr2=0x7fd704199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 0x7fd704199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 -- 172.21.15.28:0/2600533316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7041a48d0 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.158+0000 7fd703fff640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b970 tx=0x7fd6fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.159+0000 7fd700ff9640 1 -- 172.21.15.28:0/2600533316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc019070 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.159+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7041a4aa0 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.159+0000 7fd700ff9640 1 -- 172.21.15.28:0/2600533316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6fc00cb40 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.159+0000 7fd700ff9640 1 -- 172.21.15.28:0/2600533316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc007650 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.159+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7041a4f60 con 0x7fd704106a80 2024-09-19T09:31:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.161+0000 7fd700ff9640 1 -- 172.21.15.28:0/2600533316 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd6fc01c070 con 0x7fd704106a80 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.161+0000 7fd70c2dc740 1 Processor -- start 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.161+0000 7fd70c2dc740 1 -- start start 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fa9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc76f10090 con 0x55bc76fb4bb0 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc76eb7a70 con 0x55bc76fa99d0 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc76f0c050 con 0x55bc76faa2f0 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45130/0 (socket says 172.21.15.28:45130) 2024-09-19T09:31:57.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd702ffd640 1 -- 172.21.15.28:0/717069014 learned_addr learned my addr 172.21.15.28:0/717069014 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd700ff9640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 0x7fd6ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd702ffd640 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fa9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 -- 172.21.15.28:0/717069014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 msgr2=0x55bc76fa9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fa9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd703fff640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 0x7fd6ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 -- 172.21.15.28:0/717069014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 msgr2=0x55bc76fb6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 -- 172.21.15.28:0/717069014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc76ec4130 con 0x55bc76faa2f0 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd702ffd640 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd700ff9640 1 -- 172.21.15.28:0/2600533316 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd6fc0cef40 con 0x7fd704106a80 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.162+0000 7fd7037fe640 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fb4670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4005940 tx=0x7fd6f4031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20addb6c698a5609 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd703fff640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 0x7fd6ec0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc097a50 tx=0x7fd6fc06d3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd6e9ffb640 1 -- 172.21.15.28:0/717069014 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4040070 con 0x55bc76faa2f0 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd6e9ffb640 1 -- 172.21.15.28:0/717069014 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f4005500 con 0x55bc76faa2f0 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd6e9ffb640 1 -- 172.21.15.28:0/717069014 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4035040 con 0x55bc76faa2f0 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 -- 172.21.15.28:0/717069014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 msgr2=0x55bc76fb4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fb4670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4005940 tx=0x7fd6f4031630 comp rx=0 tx=0).stop 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 -- 172.21.15.28:0/717069014 shutdown_connections 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fb4670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/717069014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fa9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 -- 172.21.15.28:0/717069014 >> 172.21.15.28:0/717069014 conn(0x55bc76fa5f60 msgr2=0x55bc76fa6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 -- 172.21.15.28:0/717069014 shutdown_connections 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.163+0000 7fd70c2dc740 1 -- 172.21.15.28:0/717069014 wait complete. 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 -- start start 2024-09-19T09:31:57.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 Processor -- start 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fb4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 0x55bc76fb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fb7780 con 0x55bc76faa2f0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc76ecaf20 con 0x55bc76fa99d0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fb7900 con 0x55bc76fb4bb0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd70c2dc740 10 client.0 _ll_register_callbacks cb 0x55bc76e94030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd7037fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 0x55bc76fb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fb4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd7037fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 0x55bc76fb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45132/0 (socket says 172.21.15.28:45132) 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd7037fe640 1 -- 172.21.15.28:0/1821783715 learned_addr learned my addr 172.21.15.28:0/1821783715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:57.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.164+0000 7fd702ffd640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 -- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fb4bb0 msgr2=0x55bc76fb1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 -- 172.21.15.28:0/1821783715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 msgr2=0x55bc76fb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 0x55bc76fb1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 -- 172.21.15.28:0/1821783715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc76c812b0 con 0x55bc76fa99d0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd702ffd640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fb4330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55bc76fa72e0 tx=0x7fd6f800c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49c7406e49c378a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd6e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f801c040 con 0x55bc76fa99d0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6f4005500 con 0x55bc76fa99d0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd6f8004030 con 0x55bc76fa99d0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6f8004230 con 0x55bc76fa99d0 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd70c2dc740 10 client.24764 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:57.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.165+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc76fc3010 con 0x55bc76fa99d0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f4005500 con 0x55bc76fa99d0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd6f8022400 con 0x55bc76fa99d0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 10 client.24764 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 10 client.24764 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 20 client.24764 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc76fc55f0 0x55bc76fc6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 4 client.24764 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc76f1d600 con 0x55bc76fc55f0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.166+0000 7fd7037fe640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc76fc55f0 0x55bc76fc6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.167+0000 7fd7037fe640 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc76fc55f0 0x55bc76fc6a60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac64072404943a6d server_cookie=60809a05b3ff016 in_seq=0 out_seq=0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.167+0000 7fd6e2ffd640 10 client.24764 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.167+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/1821783715 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc76f1d600 con 0x55bc76fc55f0 2024-09-19T09:31:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.167+0000 7fd6e2ffd640 10 client.24764 handle_command_reply: tid=0 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.167+0000 7fd70c2dc740 1 client.24764 shutdown 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 msgr2=0x55bc76fb4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fb4330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55bc76fa72e0 tx=0x7fd6f800c410 comp rx=0 tx=0).stop 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 shutdown_connections 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc76fc55f0 0x55bc76fc6a60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fb4bb0 0x55bc76fb1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76faa2f0 0x55bc76fb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/1821783715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fb4330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd6e2ffd640 0 client.24764 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:31:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 >> 172.21.15.28:0/1821783715 conn(0x55bc76fa5f60 msgr2=0x55bc76fabc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 shutdown_connections 2024-09-19T09:31:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 1 -- 172.21.15.28:0/1821783715 wait complete. 2024-09-19T09:31:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.168+0000 7fd70c2dc740 20 client.24764 trim_cache size 0 max 16384 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 Processor -- start 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 -- start start 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fa99d0 0x55bc76fa0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76faa2f0 0x55bc76fb0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc4da0 con 0x55bc76fb4bb0 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc5470 con 0x55bc76faa2f0 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc5b40 con 0x55bc76fa99d0 2024-09-19T09:31:57.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45142/0 (socket says 172.21.15.28:45142) 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 -- 172.21.15.28:0/877478257 learned_addr learned my addr 172.21.15.28:0/877478257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fa99d0 0x55bc76fa0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd7037fe640 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76faa2f0 0x55bc76fb0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 -- 172.21.15.28:0/877478257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fa99d0 msgr2=0x55bc76fa0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fa99d0 0x55bc76fa0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 -- 172.21.15.28:0/877478257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76faa2f0 msgr2=0x55bc76fb0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76faa2f0 0x55bc76fb0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.186+0000 7fd702ffd640 1 -- 172.21.15.28:0/877478257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc76fa5d90 con 0x55bc76fb4bb0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd702ffd640 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8013ca0 tx=0x7fd6f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e34bc943a8058 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd6e97fa640 1 -- 172.21.15.28:0/877478257 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80043d0 con 0x55bc76fb4bb0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd6e97fa640 1 -- 172.21.15.28:0/877478257 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f8011d00 con 0x55bc76fb4bb0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd6e97fa640 1 -- 172.21.15.28:0/877478257 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8005410 con 0x55bc76fb4bb0 2024-09-19T09:31:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 -- 172.21.15.28:0/877478257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 msgr2=0x55bc76faff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8013ca0 tx=0x7fd6f800c7e0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 -- 172.21.15.28:0/877478257 shutdown_connections 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76faa2f0 0x55bc76fb0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76faff50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/877478257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76fa99d0 0x55bc76fa0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 -- 172.21.15.28:0/877478257 >> 172.21.15.28:0/877478257 conn(0x55bc76fa5f60 msgr2=0x55bc76fb1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 -- 172.21.15.28:0/877478257 shutdown_connections 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.187+0000 7fd70c2dc740 1 -- 172.21.15.28:0/877478257 wait complete. 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 -- start start 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 Processor -- start 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fc7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fc8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc68e0 con 0x55bc76fb4bb0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc6210 con 0x55bc76fa99d0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc76fc6fb0 con 0x55bc76faa2f0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd70c2dc740 10 client.0 _ll_register_callbacks cb 0x7fd6f8013b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd702ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45154/0 (socket says 172.21.15.28:45154) 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.188+0000 7fd702ffd640 1 -- 172.21.15.28:0/3411085641 learned_addr learned my addr 172.21.15.28:0/3411085641 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd7037fe640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fc8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fc7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 -- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 msgr2=0x55bc76fc8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fc8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 -- 172.21.15.28:0/3411085641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 msgr2=0x55bc76fc7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fc7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 -- 172.21.15.28:0/3411085641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7702df90 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd702ffd640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800cab0 tx=0x7fd6f80226d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d6b9e01131af94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd6e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8004000 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd6f80043d0 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6f8011d00 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6f80229f0 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd70c2dc740 10 client.15030 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc7702f290 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.189+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f8011d00 con 0x55bc76fb4bb0 2024-09-19T09:31:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd6f803a030 con 0x55bc76fb4bb0 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 10 client.15030 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 20 client.15030 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc77033b80 0x55bc77032890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 4 client.15030 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "status"}) -- 0x55bc76f17b10 con 0x55bc77033b80 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd7037fe640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc77033b80 0x55bc77032890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd7037fe640 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc77033b80 0x55bc77032890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccecc42264d1bc2a server_cookie=59922de4b1bed5ac in_seq=0 out_seq=0 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.190+0000 7fd6e2ffd640 10 client.15030 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd6e2ffd640 1 -- 172.21.15.28:0/3411085641 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55bc76f17b10 con 0x55bc77033b80 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd6e2ffd640 10 client.15030 handle_command_reply: tid=0 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 client.15030 shutdown 2024-09-19T09:31:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 msgr2=0x55bc76fc87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800cab0 tx=0x7fd6f80226d0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 shutdown_connections 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55bc77033b80 0x55bc77032890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc76fb4bb0 0x55bc76fc87b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc76faa2f0 0x55bc76fc8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 --2- 172.21.15.28:0/3411085641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc76fa99d0 0x55bc76fc7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 >> 172.21.15.28:0/3411085641 conn(0x55bc76fa5f60 msgr2=0x55bc76fb1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd6e2ffd640 0 client.15030 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 shutdown_connections 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 1 -- 172.21.15.28:0/3411085641 wait complete. 2024-09-19T09:31:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.191+0000 7fd70c2dc740 20 client.15030 trim_cache size 0 max 16384 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 2, 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24478, 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 66.034905447, 2024-09-19T09:31:57.200 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:31:57.201 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:31:57.201 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:31:57.201 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 97.422174867999999 2024-09-19T09:31:57.201 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:31:57.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 msgr2=0x7fd6ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 0x7fd6ec0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc097a50 tx=0x7fd6fc06d3c0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 msgr2=0x7fd7041a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b970 tx=0x7fd6fc00be30 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 shutdown_connections 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd6ec0a32d0 0x7fd6ec0a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd704106a80 0x7fd7041a4390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd704101420 0x7fd704199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 --2- 172.21.15.28:0/2600533316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd704100a80 0x7fd704199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 >> 172.21.15.28:0/2600533316 conn(0x7fd7040fc7a0 msgr2=0x7fd7040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 shutdown_connections 2024-09-19T09:31:57.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:57.192+0000 7fd709f03640 1 -- 172.21.15.28:0/2600533316 wait complete. 2024-09-19T09:31:57.793 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:31:58.046 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:58.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 -- 172.21.15.28:0/3070712651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 msgr2=0x7f7f68103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 --2- 172.21.15.28:0/3070712651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f68103120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009900 tx=0x7f7f6402a480 comp rx=0 tx=0).stop 2024-09-19T09:31:58.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 -- 172.21.15.28:0/3070712651 shutdown_connections 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 --2- 172.21.15.28:0/3070712651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f68103660 0x7f7f6810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 --2- 172.21.15.28:0/3070712651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6810c170 0x7f7f6810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 --2- 172.21.15.28:0/3070712651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f68103120 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 -- 172.21.15.28:0/3070712651 >> 172.21.15.28:0/3070712651 conn(0x7f7f680fc7a0 msgr2=0x7f7f680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 -- 172.21.15.28:0/3070712651 shutdown_connections 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.539+0000 7f7f705aa640 1 -- 172.21.15.28:0/3070712651 wait complete. 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 Processor -- start 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 -- start start 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 0x7f7f68198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6810c170 0x7f7f6819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68111f70 con 0x7f7f68102d40 2024-09-19T09:31:58.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68111df0 con 0x7f7f6810c170 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f705aa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681120f0 con 0x7f7f68103660 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.540+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45178/0 (socket says 172.21.15.28:45178) 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 -- 172.21.15.28:0/4160823582 learned_addr learned my addr 172.21.15.28:0/4160823582 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6810c170 0x7f7f6819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6db45640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 0x7f7f68198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 -- 172.21.15.28:0/4160823582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 msgr2=0x7f7f68198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 0x7f7f68198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 -- 172.21.15.28:0/4160823582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6810c170 msgr2=0x7f7f6819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6810c170 0x7f7f6819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 -- 172.21.15.28:0/4160823582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6819d490 con 0x7f7f68102d40 2024-09-19T09:31:58.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6db45640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 0x7f7f68198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f6e346640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006010 tx=0x7f7f640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f5f7fe640 1 -- 172.21.15.28:0/4160823582 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6403f030 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f5f7fe640 1 -- 172.21.15.28:0/4160823582 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64003ce0 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6819d720 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f5f7fe640 1 -- 172.21.15.28:0/4160823582 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64035070 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.541+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f681a5060 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.543+0000 7f7f5f7fe640 1 -- 172.21.15.28:0/4160823582 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7f64042030 con 0x7f7f68102d40 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 Processor -- start 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 -- start start 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdcedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcddbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607bcd35090 con 0x5607bcdcf2f0 2024-09-19T09:31:58.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607bccdca70 con 0x5607bcdce9d0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f72983740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607bcd31050 con 0x5607bcdd9bb0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f5f7fe640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 0x7f7f540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45194/0 (socket says 172.21.15.28:45194) 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.544+0000 7f7f6e346640 1 -- 172.21.15.28:0/62485223 learned_addr learned my addr 172.21.15.28:0/62485223 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdcedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 0x7f7f540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6db45640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcddbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f5f7fe640 1 -- 172.21.15.28:0/4160823582 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7f640f0160 con 0x7f7f68102d40 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 -- 172.21.15.28:0/62485223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 msgr2=0x5607bcddbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcddbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 -- 172.21.15.28:0/62485223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 msgr2=0x5607bcdcedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdcedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 -- 172.21.15.28:0/62485223 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607bcce9130 con 0x5607bcdcf2f0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdcedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6e346640 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f64084530 tx=0x7f7f640b82e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8617dc8c78c3fb34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 0x7f7f540a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009370 tx=0x7f7f60009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f5cff9640 1 -- 172.21.15.28:0/62485223 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64049070 con 0x5607bcdcf2f0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f5cff9640 1 -- 172.21.15.28:0/62485223 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f640bcdb0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.545+0000 7f7f5cff9640 1 -- 172.21.15.28:0/62485223 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640c0a60 con 0x5607bcdcf2f0 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 -- 172.21.15.28:0/62485223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 msgr2=0x5607bcdd9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f64084530 tx=0x7f7f640b82e0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 -- 172.21.15.28:0/62485223 shutdown_connections 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcddbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd9670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 --2- 172.21.15.28:0/62485223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdcedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 -- 172.21.15.28:0/62485223 >> 172.21.15.28:0/62485223 conn(0x5607bcdcaf60 msgr2=0x5607bcdcb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 -- 172.21.15.28:0/62485223 shutdown_connections 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.546+0000 7f7f72983740 1 -- 172.21.15.28:0/62485223 wait complete. 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 -- start start 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 Processor -- start 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdd9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607bcddc540 con 0x5607bcdcf2f0 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607bcceff20 con 0x5607bcdce9d0 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607bcddc6c0 con 0x5607bcdd9bb0 2024-09-19T09:31:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f72983740 10 client.0 _ll_register_callbacks cb 0x5607bccb9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45202/0 (socket says 172.21.15.28:45202) 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 -- 172.21.15.28:0/503529753 learned_addr learned my addr 172.21.15.28:0/503529753 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6e346640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdd9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 -- 172.21.15.28:0/503529753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 msgr2=0x5607bcdd6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 -- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 msgr2=0x5607bcdd9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdd9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.547+0000 7f7f6db45640 1 -- 172.21.15.28:0/503529753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607bcaa62b0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f6db45640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f58013710 tx=0x7f7f58013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4c689ae5a165d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:58.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58011dd0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7f5801b3e0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f640bcdb0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f5801b620 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 10 client.15042 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607bcde7de0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f640bcdb0 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7f5801a070 con 0x5607bcdcf2f0 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 20 client.15042 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bcdea4a0 0x5607bcdeb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 4 client.15042 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.548+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607bcd42600 con 0x5607bcdea4a0 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.549+0000 7f7f6e346640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bcdea4a0 0x5607bcdeb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.549+0000 7f7f6e346640 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bcdea4a0 0x5607bcdeb910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d33570b3664789e server_cookie=1074b8eb7c5e491b in_seq=0 out_seq=0 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.549+0000 7f7f51ffb640 10 client.15042 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f51ffb640 1 -- 172.21.15.28:0/503529753 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607bcd42600 con 0x5607bcdea4a0 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f51ffb640 10 client.15042 handle_command_reply: tid=0 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 client.15042 shutdown 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 msgr2=0x5607bcdd67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f58013710 tx=0x7f7f58013bd0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 shutdown_connections 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd67d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bcdea4a0 0x5607bcdeb910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 --2- 172.21.15.28:0/503529753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdce9d0 0x5607bcdd9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f51ffb640 0 client.15042 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 >> 172.21.15.28:0/503529753 conn(0x5607bcdcaf60 msgr2=0x5607bcdd0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 shutdown_connections 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 1 -- 172.21.15.28:0/503529753 wait complete. 2024-09-19T09:31:58.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.550+0000 7f7f72983740 20 client.15042 trim_cache size 0 max 16384 2024-09-19T09:31:58.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 Processor -- start 2024-09-19T09:31:58.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 -- start start 2024-09-19T09:31:58.570 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:31:58 smithi028 ceph-mon[30972]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 25 B/s, 0 objects/s recovering 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 0x5607bcdf3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcde5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607bcde9c50 con 0x5607bcdce9d0 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607bcdea320 con 0x5607bcdcf2f0 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f72983740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607bcdea9f0 con 0x5607bcdd9bb0 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45218/0 (socket says 172.21.15.28:45218) 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.568+0000 7f7f6e346640 1 -- 172.21.15.28:0/2995970450 learned_addr learned my addr 172.21.15.28:0/2995970450 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6db45640 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 0x5607bcdf3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 -- 172.21.15.28:0/2995970450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 msgr2=0x5607bcde5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcde5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 -- 172.21.15.28:0/2995970450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 msgr2=0x5607bcdf3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 0x5607bcdf3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 -- 172.21.15.28:0/2995970450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607bcdcad90 con 0x5607bcdce9d0 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6db45640 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 0x5607bcdf3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f6e346640 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f640c5320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19046fdc008644e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f53fff640 1 -- 172.21.15.28:0/2995970450 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64049040 con 0x5607bcdce9d0 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f53fff640 1 -- 172.21.15.28:0/2995970450 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f640da070 con 0x5607bcdce9d0 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.569+0000 7f7f53fff640 1 -- 172.21.15.28:0/2995970450 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640edd30 con 0x5607bcdce9d0 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- 172.21.15.28:0/2995970450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 msgr2=0x5607bcdd56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f640c5320 comp rx=0 tx=0).stop 2024-09-19T09:31:58.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- 172.21.15.28:0/2995970450 shutdown_connections 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdcf2f0 0x5607bcdf3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdd9bb0 0x5607bcde5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 --2- 172.21.15.28:0/2995970450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd56a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- 172.21.15.28:0/2995970450 >> 172.21.15.28:0/2995970450 conn(0x5607bcdcaf60 msgr2=0x5607bcdd4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- 172.21.15.28:0/2995970450 shutdown_connections 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- 172.21.15.28:0/2995970450 wait complete. 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.570+0000 7f7f72983740 1 -- start start 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 Processor -- start 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607bcdeb790 con 0x5607bcdce9d0 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607bcdeb0c0 con 0x5607bcdd9bb0 2024-09-19T09:31:58.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607bcdebe60 con 0x5607bcdcf2f0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f72983740 10 client.0 _ll_register_callbacks cb 0x7f7f58009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45222/0 (socket says 172.21.15.28:45222) 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 -- 172.21.15.28:0/3967240130 learned_addr learned my addr 172.21.15.28:0/3967240130 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6db45640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6eb47640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 -- 172.21.15.28:0/3967240130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdcf2f0 msgr2=0x5607bcdd2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 -- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdd9bb0 msgr2=0x5607bcdd3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.571+0000 7f7f6e346640 1 -- 172.21.15.28:0/3967240130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607bcdd3850 con 0x5607bcdce9d0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f6e346640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f640058e0 tx=0x7f7f640edaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55e78ad85131601 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:58.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64049040 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7f640ba9b0 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f640da070 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f640c0a60 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 10 client.15048 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607bce57c90 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f640da070 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7f640c4e90 con 0x5607bcdce9d0 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 20 client.15048 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.572+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bce5c2f0 0x5607bce5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f72983740 4 client.15048 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:31:58.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "status"}) -- 0x5607bcd3cb10 con 0x5607bce5c2f0 2024-09-19T09:31:58.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f6db45640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bce5c2f0 0x5607bce5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:58.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f6db45640 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bce5c2f0 0x5607bce5b000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad601ca7cc74ff98 server_cookie=1c5277d5632942eb in_seq=0 out_seq=0 2024-09-19T09:31:58.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f51ffb640 10 client.15048 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f51ffb640 1 -- 172.21.15.28:0/3967240130 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5607bcd3cb10 con 0x5607bce5c2f0 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.573+0000 7f7f51ffb640 10 client.15048 handle_command_reply: tid=0 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 client.15048 shutdown 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 msgr2=0x5607bcdd4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f640058e0 tx=0x7f7f640edaa0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 shutdown_connections 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5607bce5c2f0 0x5607bce5b000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607bcdd9bb0 0x5607bcdd3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f51ffb640 0 client.15048 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607bcdcf2f0 0x5607bcdd2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 --2- 172.21.15.28:0/3967240130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607bcdce9d0 0x5607bcdd4420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 >> 172.21.15.28:0/3967240130 conn(0x5607bcdcaf60 msgr2=0x5607bcdd57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 shutdown_connections 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 1 -- 172.21.15.28:0/3967240130 wait complete. 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f72983740 20 client.15048 trim_cache size 0 max 16384 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 1, 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24482, 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:31:58.581 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 68.427865589999996, 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 85.498491977 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 msgr2=0x7f7f540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 0x7f7f540a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009370 tx=0x7f7f60009240 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 msgr2=0x7f7f681984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.574+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006010 tx=0x7f7f640042f0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 shutdown_connections 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7f540a32d0 0x7f7f540a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6810c170 0x7f7f6819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f68103660 0x7f7f68198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 --2- 172.21.15.28:0/4160823582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f68102d40 0x7f7f681984e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 >> 172.21.15.28:0/4160823582 conn(0x7f7f680fc7a0 msgr2=0x7f7f680fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 shutdown_connections 2024-09-19T09:31:58.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:58.575+0000 7f7f705aa640 1 -- 172.21.15.28:0/4160823582 wait complete. 2024-09-19T09:31:58.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:31:58 smithi078 ceph-mon[35619]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 25 B/s, 0 objects/s recovering 2024-09-19T09:31:58.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:31:58 smithi038 ceph-mon[35271]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 25 B/s, 0 objects/s recovering 2024-09-19T09:31:59.234 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:31:59.494 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:31:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 -- 172.21.15.28:0/1347064784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4106a80 msgr2=0x7ff3b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 --2- 172.21.15.28:0/1347064784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4106a80 0x7ff3b4106e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009930 tx=0x7ff3b002a420 comp rx=0 tx=0).stop 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 -- 172.21.15.28:0/1347064784 shutdown_connections 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 --2- 172.21.15.28:0/1347064784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4100a80 0x7ff3b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 --2- 172.21.15.28:0/1347064784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4101420 0x7ff3b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 --2- 172.21.15.28:0/1347064784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4106a80 0x7ff3b4106e60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 -- 172.21.15.28:0/1347064784 >> 172.21.15.28:0/1347064784 conn(0x7ff3b40fc7a0 msgr2=0x7ff3b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 -- 172.21.15.28:0/1347064784 shutdown_connections 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.932+0000 7ff3bb599640 1 -- 172.21.15.28:0/1347064784 wait complete. 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 Processor -- start 2024-09-19T09:31:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 -- start start 2024-09-19T09:31:59.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4101420 0x7ff3b4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4106a80 0x7ff3b419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4110e80 con 0x7ff3b4100a80 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4110d00 con 0x7ff3b4101420 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3bb599640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b4111000 con 0x7ff3b4106a80 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45250/0 (socket says 172.21.15.28:45250) 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b9335640 1 -- 172.21.15.28:0/2907427070 learned_addr learned my addr 172.21.15.28:0/2907427070 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4106a80 0x7ff3b419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4101420 0x7ff3b4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.933+0000 7ff3b9335640 1 -- 172.21.15.28:0/2907427070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4106a80 msgr2=0x7ff3b419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4106a80 0x7ff3b419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3b9335640 1 -- 172.21.15.28:0/2907427070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4101420 msgr2=0x7ff3b4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4101420 0x7ff3b4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3b9335640 1 -- 172.21.15.28:0/2907427070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b419c2d0 con 0x7ff3b4100a80 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009230 tx=0x7ff3b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3aa7fc640 1 -- 172.21.15.28:0/2907427070 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b003f030 con 0x7ff3b4100a80 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b419c560 con 0x7ff3b4100a80 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3aa7fc640 1 -- 172.21.15.28:0/2907427070 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b0003ad0 con 0x7ff3b4100a80 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3aa7fc640 1 -- 172.21.15.28:0/2907427070 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0035070 con 0x7ff3b4100a80 2024-09-19T09:31:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.934+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b406a790 con 0x7ff3b4100a80 2024-09-19T09:31:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.937+0000 7ff3bd972740 1 Processor -- start 2024-09-19T09:31:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.937+0000 7ff3aa7fc640 1 -- 172.21.15.28:0/2907427070 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff3b0042030 con 0x7ff3b4100a80 2024-09-19T09:31:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.937+0000 7ff3bd972740 1 -- start start 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5634529cd090 con 0x563452a669d0 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563452974a70 con 0x563452a71bb0 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3bd972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634529c9050 con 0x563452a672f0 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36504/0 (socket says 172.21.15.28:36504) 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 -- 172.21.15.28:0/1824096659 learned_addr learned my addr 172.21.15.28:0/1824096659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3aa7fc640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 0x7ff3a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 -- 172.21.15.28:0/1824096659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 msgr2=0x563452a73fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a73fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.938+0000 7ff3b9335640 1 -- 172.21.15.28:0/1824096659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 msgr2=0x563452a66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9335640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9335640 1 -- 172.21.15.28:0/1824096659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563452981130 con 0x563452a672f0 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3aa7fc640 1 -- 172.21.15.28:0/2907427070 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff3b00eff80 con 0x7ff3b4100a80 2024-09-19T09:31:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 0x7ff3a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9335640 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ed8a0 tx=0x7ff3b004b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b620befb6778a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff39f7fe640 1 -- 172.21.15.28:0/1824096659 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00bbfb0 con 0x563452a672f0 2024-09-19T09:31:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff39f7fe640 1 -- 172.21.15.28:0/1824096659 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b007cfe0 con 0x563452a672f0 2024-09-19T09:31:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff39f7fe640 1 -- 172.21.15.28:0/1824096659 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00bbdd0 con 0x563452a672f0 2024-09-19T09:31:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.939+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 0x7ff3a00a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007f80 tx=0x7ff3ac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- 172.21.15.28:0/1824096659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 msgr2=0x563452a71670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ed8a0 tx=0x7ff3b004b000 comp rx=0 tx=0).stop 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- 172.21.15.28:0/1824096659 shutdown_connections 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a672f0 0x563452a71670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 --2- 172.21.15.28:0/1824096659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- 172.21.15.28:0/1824096659 >> 172.21.15.28:0/1824096659 conn(0x563452a62f60 msgr2=0x563452a63340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- 172.21.15.28:0/1824096659 shutdown_connections 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- 172.21.15.28:0/1824096659 wait complete. 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.940+0000 7ff3bd972740 1 -- start start 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 Processor -- start 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a6e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563452a74540 con 0x563452a669d0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563452987f20 con 0x563452a672f0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563452a746c0 con 0x563452a71bb0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3bd972740 10 client.0 _ll_register_callbacks cb 0x563452951030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45272/0 (socket says 172.21.15.28:45272) 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3b9335640 1 -- 172.21.15.28:0/391970395 learned_addr learned my addr 172.21.15.28:0/391970395 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.941+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a6e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 -- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 msgr2=0x563452a6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 -- 172.21.15.28:0/391970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 msgr2=0x563452a6e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a6e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 -- 172.21.15.28:0/391970395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56345273e2b0 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3b9335640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0000c00 tx=0x7ff3b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fff186bc97b899 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0049070 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff3b00edbb0 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5634529da600 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3b00bbfb0 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3bd972740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563452a74540 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.942+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5634529da600 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b0003c80 con 0x563452a669d0 2024-09-19T09:31:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 20 client.15060 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452a82110 0x563452a83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 4 client.15060 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634529d4b10 con 0x563452a82110 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452a82110 0x563452a83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452a82110 0x563452a83580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59eb807400cfe05d server_cookie=1a72e7ef71e6f683 in_seq=0 out_seq=0 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.943+0000 7ff39cff9640 10 client.15060 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff39cff9640 1 -- 172.21.15.28:0/391970395 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634529d4b10 con 0x563452a82110 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff39cff9640 10 client.15060 handle_command_reply: tid=0 2024-09-19T09:31:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 client.15060 shutdown 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 msgr2=0x563452a71430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0000c00 tx=0x7ff3b002a420 comp rx=0 tx=0).stop 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 shutdown_connections 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a6e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452a82110 0x563452a83580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 --2- 172.21.15.28:0/391970395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a71430 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 >> 172.21.15.28:0/391970395 conn(0x563452a62f60 msgr2=0x563452a68c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.944+0000 7ff39cff9640 0 client.15060 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.945+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 shutdown_connections 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.945+0000 7ff3bd972740 1 -- 172.21.15.28:0/391970395 wait complete. 2024-09-19T09:31:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.945+0000 7ff3bd972740 20 client.15060 trim_cache size 0 max 16384 2024-09-19T09:31:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.962+0000 7ff3bd972740 1 Processor -- start 2024-09-19T09:31:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.962+0000 7ff3bd972740 1 -- start start 2024-09-19T09:31:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a669d0 0x563452a81a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563452a818c0 con 0x563452a672f0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563452a81f90 con 0x563452a71bb0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3bd972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563452a82660 con 0x563452a669d0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45288/0 (socket says 172.21.15.28:45288) 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 -- 172.21.15.28:0/104874991 learned_addr learned my addr 172.21.15.28:0/104874991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b9335640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a669d0 0x563452a81a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 -- 172.21.15.28:0/104874991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a669d0 msgr2=0x563452a81a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a669d0 0x563452a81a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 -- 172.21.15.28:0/104874991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 msgr2=0x563452a6e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a6e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 -- 172.21.15.28:0/104874991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563452a66800 con 0x563452a672f0 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x563452a642e0 tx=0x7ff3a400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d52d00eda8de37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff39effd640 1 -- 172.21.15.28:0/104874991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4003de0 con 0x563452a672f0 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff39effd640 1 -- 172.21.15.28:0/104874991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a401d910 con 0x563452a672f0 2024-09-19T09:31:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.963+0000 7ff39effd640 1 -- 172.21.15.28:0/104874991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011b20 con 0x563452a672f0 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- 172.21.15.28:0/104874991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 msgr2=0x563452a82eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x563452a642e0 tx=0x7ff3a400bd90 comp rx=0 tx=0).stop 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- 172.21.15.28:0/104874991 shutdown_connections 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a71bb0 0x563452a6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a672f0 0x563452a82eb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 --2- 172.21.15.28:0/104874991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a669d0 0x563452a81a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- 172.21.15.28:0/104874991 >> 172.21.15.28:0/104874991 conn(0x563452a62f60 msgr2=0x563452a6e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- 172.21.15.28:0/104874991 shutdown_connections 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- 172.21.15.28:0/104874991 wait complete. 2024-09-19T09:31:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 -- start start 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.964+0000 7ff3bd972740 1 Processor -- start 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a84bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563452a83400 con 0x563452a669d0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563452a82d30 con 0x563452a672f0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563452a83ad0 con 0x563452a71bb0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3bd972740 10 client.0 _ll_register_callbacks cb 0x7ff3a400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b8b34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a84bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45300/0 (socket says 172.21.15.28:45300) 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 -- 172.21.15.28:0/2745968328 learned_addr learned my addr 172.21.15.28:0/2745968328 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:31:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9b36640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 -- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 msgr2=0x563452a85110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a85110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 -- 172.21.15.28:0/2745968328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 msgr2=0x563452a84bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a84bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.965+0000 7ff3b9335640 1 -- 172.21.15.28:0/2745968328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563452aea810 con 0x563452a669d0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3b9335640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00bc400 tx=0x7ff3b0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f02425b7eb5739 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0049070 con 0x563452a669d0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff3b003b460 con 0x563452a669d0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a401d910 con 0x563452a669d0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3b00bcb70 con 0x563452a669d0 2024-09-19T09:31:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 10 client.15066 fetch_fsmap learned FSMap version 26 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563452aea810 con 0x563452a669d0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a401d910 con 0x563452a669d0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b003bc30 con 0x563452a669d0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 20 client.15066 populate_metadata read hostname 'smithi028' 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452af01b0 0x563452aeeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 4 client.15066 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.966+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "status"}) -- 0x5634529510e0 con 0x563452af01b0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452af01b0 0x563452aeeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff3b8b34640 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452af01b0 0x563452aeeec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b02f588d29a9a2 server_cookie=7fc7fc5f45f59d6c in_seq=0 out_seq=0 2024-09-19T09:31:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff39cff9640 10 client.15066 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff39cff9640 1 -- 172.21.15.28:0/2745968328 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5634529510e0 con 0x563452af01b0 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff39cff9640 10 client.15066 handle_command_reply: tid=0 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.967+0000 7ff3bd972740 1 client.15066 shutdown 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 msgr2=0x563452a84690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00bc400 tx=0x7ff3b0009620 comp rx=0 tx=0).stop 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 shutdown_connections 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563452a672f0 0x563452a84bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563452af01b0 0x563452aeeec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563452a71bb0 0x563452a85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 --2- 172.21.15.28:0/2745968328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563452a669d0 0x563452a84690 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 >> 172.21.15.28:0/2745968328 conn(0x563452a62f60 msgr2=0x563452a69e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff39cff9640 0 client.15066 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 shutdown_connections 2024-09-19T09:31:59.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 1 -- 172.21.15.28:0/2745968328 wait complete. 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bd972740 20 client.15066 trim_cache size 0 max 16384 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24499, 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:31:59.975 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 76.414650820999995, 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 95.253482914000003 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:31:59.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 msgr2=0x7ff3a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 0x7ff3a00a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007f80 tx=0x7ff3ac004a40 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 msgr2=0x7ff3b4197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009230 tx=0x7ff3b0004270 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 shutdown_connections 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.968+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff3a00a32d0 0x7ff3a00a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3b4101420 0x7ff3b4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3b4106a80 0x7ff3b419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 --2- 172.21.15.28:0/2907427070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3b4100a80 0x7ff3b4197320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 >> 172.21.15.28:0/2907427070 conn(0x7ff3b40fc7a0 msgr2=0x7ff3b40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 shutdown_connections 2024-09-19T09:31:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:31:59.969+0000 7ff3bb599640 1 -- 172.21.15.28:0/2907427070 wait complete. 2024-09-19T09:32:00.647 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:00 smithi028 ceph-mon[30972]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 22 B/s, 0 objects/s recovering 2024-09-19T09:32:00.676 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T09:32:00.676 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:32:00.689 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:32:00.689 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T09:32:00.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:00 smithi078 ceph-mon[35619]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 22 B/s, 0 objects/s recovering 2024-09-19T09:32:00.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:00 smithi038 ceph-mon[35271]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 22 B/s, 0 objects/s recovering 2024-09-19T09:32:00.947 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- 172.21.15.28:0/2883158586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb768102660 msgr2=0x7fb768104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- 172.21.15.28:0/2883158586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb768102660 0x7fb768104a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb758009ca0 tx=0x7fb75802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- 172.21.15.28:0/2883158586 shutdown_connections 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- 172.21.15.28:0/2883158586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7681078c0 0x7fb768109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- 172.21.15.28:0/2883158586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 0x7fb768107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- 172.21.15.28:0/2883158586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb768102660 0x7fb768104a50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- 172.21.15.28:0/2883158586 >> 172.21.15.28:0/2883158586 conn(0x7fb7680fc280 msgr2=0x7fb7680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- 172.21.15.28:0/2883158586 shutdown_connections 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- 172.21.15.28:0/2883158586 wait complete. 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 Processor -- start 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- start start 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768102660 0x7fb768197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 0x7fb768198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810d890 con 0x7fb7681078c0 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810d710 con 0x7fb768102660 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.399+0000 7fb76faab640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810da10 con 0x7fb768104f90 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:01.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45308/0 (socket says 172.21.15.28:45308) 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76d847640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768102660 0x7fb768197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 -- 172.21.15.28:0/1054820906 learned_addr learned my addr 172.21.15.28:0/1054820906 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76d046640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 0x7fb768198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 -- 172.21.15.28:0/1054820906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 msgr2=0x7fb768198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 0x7fb768198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 -- 172.21.15.28:0/1054820906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768102660 msgr2=0x7fb768197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768102660 0x7fb768197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 -- 172.21.15.28:0/1054820906 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76819fb50 con 0x7fb7681078c0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76e048640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e970 tx=0x7fb76000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76000ccc0 con 0x7fb7681078c0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76000ce60 con 0x7fb7681078c0 2024-09-19T09:32:01.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760010490 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76819fe40 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.400+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a02a0 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.401+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7680fdd70 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.403+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb760005000 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.403+0000 7fb766ffd640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 0x7fb7500a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.403+0000 7fb76d847640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 0x7fb7500a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.403+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb7600cfee0 con 0x7fb7681078c0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.404+0000 7fb76d847640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 0x7fb7500a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb758000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:01.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.408+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb76009cad0 con 0x7fb7681078c0 2024-09-19T09:32:01.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.600+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb768062630 con 0x7fb7500a3300 2024-09-19T09:32:01.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.601+0000 7fb766ffd640 1 -- 172.21.15.28:0/1054820906 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb768062630 con 0x7fb7500a3300 2024-09-19T09:32:01.602 INFO:teuthology.orchestra.run.smithi028.stdout:Backend: cephadm 2024-09-19T09:32:01.602 INFO:teuthology.orchestra.run.smithi028.stdout:Available: Yes 2024-09-19T09:32:01.602 INFO:teuthology.orchestra.run.smithi028.stdout:Paused: No 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 msgr2=0x7fb7500a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 0x7fb7500a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb758000f40 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 msgr2=0x7fb76819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e970 tx=0x7fb76000ed70 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 shutdown_connections 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb7500a3300 0x7fb7500a57c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7681078c0 0x7fb76819f4e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb768104f90 0x7fb768198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 --2- 172.21.15.28:0/1054820906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb768102660 0x7fb768197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 >> 172.21.15.28:0/1054820906 conn(0x7fb7680fc280 msgr2=0x7fb768102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:01.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 shutdown_connections 2024-09-19T09:32:01.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:01.603+0000 7fb76faab640 1 -- 172.21.15.28:0/1054820906 wait complete. 2024-09-19T09:32:02.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T09:32:02.384 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:02.758 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:02 smithi028 ceph-mon[30972]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:02.758 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:02 smithi028 ceph-mon[30972]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:02.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 -- 172.21.15.28:0/4047245770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc102690 msgr2=0x7f11bc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:02.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 --2- 172.21.15.28:0/4047245770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc102690 0x7f11bc108f20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 -- 172.21.15.28:0/4047245770 shutdown_connections 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 --2- 172.21.15.28:0/4047245770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc101cd0 0x7f11bc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 --2- 172.21.15.28:0/4047245770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc102690 0x7f11bc108f20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 --2- 172.21.15.28:0/4047245770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc100ad0 0x7f11bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 -- 172.21.15.28:0/4047245770 >> 172.21.15.28:0/4047245770 conn(0x7f11bc0fc280 msgr2=0x7f11bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 -- 172.21.15.28:0/4047245770 shutdown_connections 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.839+0000 7f11c2857640 1 -- 172.21.15.28:0/4047245770 wait complete. 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 Processor -- start 2024-09-19T09:32:02.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 -- start start 2024-09-19T09:32:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc101cd0 0x7f11bc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 0x7f11bc19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10d830 con 0x7f11bc100ad0 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10d6b0 con 0x7f11bc102690 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c2857640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10d9b0 con 0x7f11bc101cd0 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45326/0 (socket says 172.21.15.28:45326) 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 -- 172.21.15.28:0/2096097543 learned_addr learned my addr 172.21.15.28:0/2096097543 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bb7fe640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc101cd0 0x7f11bc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c0df4640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 0x7f11bc19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 -- 172.21.15.28:0/2096097543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc101cd0 msgr2=0x7f11bc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc101cd0 0x7f11bc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 -- 172.21.15.28:0/2096097543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 msgr2=0x7f11bc19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 0x7f11bc19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11bbfff640 1 -- 172.21.15.28:0/2096097543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19fab0 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.840+0000 7f11c0df4640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 0x7f11bc19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11bbfff640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00d950 tx=0x7f11ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac017020 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac009c30 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19fda0 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00f720 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.841+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a0250 con 0x7f11bc100ad0 2024-09-19T09:32:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.842+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc107e70 con 0x7f11bc100ad0 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.843+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f11ac01a020 con 0x7f11bc100ad0 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.844+0000 7f11b97fa640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 0x7f11a40a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.844+0000 7f11bb7fe640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 0x7f11a40a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.844+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f11ac0cf930 con 0x7f11bc100ad0 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.844+0000 7f11bb7fe640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 0x7f11a40a5840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f11bc1993a0 tx=0x7f11b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:02.848+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f11ac021020 con 0x7f11bc100ad0 2024-09-19T09:32:02.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:02 smithi078 ceph-mon[35619]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:02.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:02 smithi078 ceph-mon[35619]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:02.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:02 smithi038 ceph-mon[35271]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:02.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:02 smithi038 ceph-mon[35271]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:03.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.035+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f11bc063020 con 0x7f11a40a3380 2024-09-19T09:32:03.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.047+0000 7f11b97fa640 1 -- 172.21.15.28:0/2096097543 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f11bc063020 con 0x7f11a40a3380 2024-09-19T09:32:03.048 INFO:teuthology.orchestra.run.smithi028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T09:32:03.048 INFO:teuthology.orchestra.run.smithi028.stdout:mds.a smithi028 running (92s) 20s ago 92s 16.8M - 19.1.1-361-g2f599a46 d2baa855c067 71bbc033a836 2024-09-19T09:32:03.048 INFO:teuthology.orchestra.run.smithi028.stdout:mds.b smithi038 running (90s) 23s ago 89s 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 e5d3a1618d5e 2024-09-19T09:32:03.048 INFO:teuthology.orchestra.run.smithi028.stdout:mds.c smithi078 running (103s) 23s ago 103s 19.7M - 19.1.1-361-g2f599a46 d2baa855c067 cd29df972961 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.d smithi028 running (111s) 20s ago 110s 14.4M - 19.1.1-361-g2f599a46 d2baa855c067 526b7f0d8e5c 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.e smithi038 running (108s) 23s ago 108s 16.9M - 19.1.1-361-g2f599a46 d2baa855c067 2d15cc925a4c 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.f smithi078 running (106s) 23s ago 105s 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 29fae996f6b6 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.g smithi028 running (96s) 20s ago 95s 16.8M - 19.1.1-361-g2f599a46 d2baa855c067 deb5bd5028b3 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.h smithi038 running (101s) 23s ago 100s 20.4M - 19.1.1-361-g2f599a46 d2baa855c067 fdff1c820477 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.i smithi078 running (98s) 23s ago 98s 26.6M - 19.1.1-361-g2f599a46 d2baa855c067 f2f5fc700b90 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.j smithi028 running (114s) 20s ago 114s 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 5b494458b16d 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.k smithi038 running (117s) 23s ago 117s 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 53d58698adb1 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mds.l smithi078 running (2m) 23s ago 119s 20.4M - 19.1.1-361-g2f599a46 d2baa855c067 0c2a7c0bcdd6 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.x smithi028 *:9283,8765 running (8m) 20s ago 8m 509M - 19.1.1-361-g2f599a46 d2baa855c067 598dadb37a24 2024-09-19T09:32:03.049 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.y smithi038 *:8443,8765 running (7m) 23s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 671ba97058b8 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:mgr.z smithi078 *:8443,8765 running (7m) 23s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 d824d11d09e2 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:mon.a smithi028 running (8m) 20s ago 9m 49.4M 2048M 19.1.1-361-g2f599a46 d2baa855c067 5e0e81b5a83f 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:mon.b smithi038 running (7m) 23s ago 7m 40.6M 2048M 19.1.1-361-g2f599a46 d2baa855c067 9fa06e52672b 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:mon.c smithi078 running (7m) 23s ago 7m 40.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 819d566cd6fe 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 smithi028 running (6m) 20s ago 6m 48.7M 4273M 19.1.1-361-g2f599a46 d2baa855c067 9a4fc946c055 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 smithi038 running (6m) 23s ago 6m 51.1M 4273M 19.1.1-361-g2f599a46 d2baa855c067 4b8455f6c4fd 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 smithi078 running (6m) 23s ago 6m 60.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1ccbb6327307 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 smithi028 running (5m) 20s ago 5m 46.2M 4273M 19.1.1-361-g2f599a46 d2baa855c067 55f2fe4a0e53 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 smithi038 running (5m) 23s ago 5m 46.2M 4273M 19.1.1-361-g2f599a46 d2baa855c067 2a81369d1ffc 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 smithi078 running (4m) 23s ago 4m 46.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 96eedc02a305 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6 smithi028 running (4m) 20s ago 4m 43.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 b8ef7a90a2f4 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7 smithi038 running (4m) 23s ago 3m 54.0M 4273M 19.1.1-361-g2f599a46 d2baa855c067 cf018c9ecf7b 2024-09-19T09:32:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8 smithi078 running (3m) 23s ago 3m 54.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 6d3635534e64 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9 smithi028 running (3m) 20s ago 3m 51.3M 4273M 19.1.1-361-g2f599a46 d2baa855c067 0a0908ae4a92 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10 smithi038 running (2m) 23s ago 2m 53.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 c34e86b86116 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11 smithi078 running (2m) 23s ago 2m 42.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 eea0baf1c7f5 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 msgr2=0x7f11a40a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 0x7f11a40a5840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f11bc1993a0 tx=0x7f11b000b390 comp rx=0 tx=0).stop 2024-09-19T09:32:03.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 msgr2=0x7f11bc197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00d950 tx=0x7f11ac00de10 comp rx=0 tx=0).stop 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 shutdown_connections 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a40a3380 0x7f11a40a5840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11bc102690 0x7f11bc19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11bc101cd0 0x7f11bc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 --2- 172.21.15.28:0/2096097543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11bc100ad0 0x7f11bc197e80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 >> 172.21.15.28:0/2096097543 conn(0x7f11bc0fc280 msgr2=0x7f11bc105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 shutdown_connections 2024-09-19T09:32:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:03.050+0000 7f11c2857640 1 -- 172.21.15.28:0/2096097543 wait complete. 2024-09-19T09:32:03.695 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:03 smithi028 ceph-mon[30972]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:03.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T09:32:03.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:03 smithi078 ceph-mon[35619]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:03.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:03 smithi038 ceph-mon[35271]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:03.980 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:04.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 -- 172.21.15.28:0/1389760612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d41026d0 msgr2=0x7f61d4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:04.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 --2- 172.21.15.28:0/1389760612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d41026d0 0x7f61d4108f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009ca0 tx=0x7f61cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 -- 172.21.15.28:0/1389760612 shutdown_connections 2024-09-19T09:32:04.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 --2- 172.21.15.28:0/1389760612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d41026d0 0x7f61d4108f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 --2- 172.21.15.28:0/1389760612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d4101d10 0x7f61d4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 --2- 172.21.15.28:0/1389760612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4100b10 0x7f61d4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 -- 172.21.15.28:0/1389760612 >> 172.21.15.28:0/1389760612 conn(0x7f61d40fc2a0 msgr2=0x7f61d40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 -- 172.21.15.28:0/1389760612 shutdown_connections 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 -- 172.21.15.28:0/1389760612 wait complete. 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.465+0000 7f61da5fd640 1 Processor -- start 2024-09-19T09:32:04.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 -- start start 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 0x7f61d419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 0x7f61d41a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410d8c0 con 0x7f61d4100b10 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410d740 con 0x7f61d4101d10 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61da5fd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410da40 con 0x7f61d41026d0 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53156/0 (socket says 172.21.15.28:53156) 2024-09-19T09:32:04.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 -- 172.21.15.28:0/3136924432 learned_addr learned my addr 172.21.15.28:0/3136924432 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d8b9a640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 0x7f61d41a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d37fe640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 0x7f61d419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 -- 172.21.15.28:0/3136924432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 msgr2=0x7f61d41a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 0x7f61d41a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 -- 172.21.15.28:0/3136924432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 msgr2=0x7f61d419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 0x7f61d419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d3fff640 1 -- 172.21.15.28:0/3136924432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d41a1ca0 con 0x7f61d4100b10 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d8b9a640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 0x7f61d41a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.466+0000 7f61d37fe640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 0x7f61d419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61d3fff640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c970 tx=0x7f61c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007c10 con 0x7f61d4100b10 2024-09-19T09:32:04.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d41a1f90 con 0x7f61d4100b10 2024-09-19T09:32:04.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61c4007db0 con 0x7f61d4100b10 2024-09-19T09:32:04.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c400f8a0 con 0x7f61d4100b10 2024-09-19T09:32:04.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.467+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d41a23e0 con 0x7f61d4100b10 2024-09-19T09:32:04.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.468+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d41a26a0 con 0x7f61d4100b10 2024-09-19T09:32:04.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.469+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f61c400fa40 con 0x7f61d4100b10 2024-09-19T09:32:04.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.470+0000 7f61d17fa640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 0x7f61bc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:04.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.470+0000 7f61d37fe640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 0x7f61bc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:04.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.470+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f61c40cfe30 con 0x7f61d4100b10 2024-09-19T09:32:04.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.470+0000 7f61d37fe640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 0x7f61bc0a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c8005fd0 tx=0x7f61c8005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:04.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.474+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61c409ca50 con 0x7f61d4100b10 2024-09-19T09:32:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.659+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f61d41a2870 con 0x7f61bc0a3300 2024-09-19T09:32:04.660 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:04 smithi028 ceph-mon[30972]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.665+0000 7f61d17fa640 1 -- 172.21.15.28:0/3136924432 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f61d41a2870 con 0x7f61bc0a3300 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:mds.all 12/12 25s ago 2m smithi028=a;smithi028=d;smithi028=g;smithi028=j;smithi038=b;smithi038=e;smithi038=h;smithi038=k;smithi078=c;smithi078=f;smithi078=i;smithi078=l;count:12 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:mgr 3/3 25s ago 7m smithi028=x;smithi038=y;smithi078=z;count:3 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:mon 3/3 25s ago 7m smithi028:172.21.15.28=a;smithi038:172.21.15.38=b;smithi078:172.21.15.78=c;count:3 2024-09-19T09:32:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:osd 12 25s ago - 2024-09-19T09:32:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 msgr2=0x7f61bc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 0x7f61bc0a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c8005fd0 tx=0x7f61c8005ea0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 msgr2=0x7f61d419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c970 tx=0x7f61c400ce30 comp rx=0 tx=0).stop 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 shutdown_connections 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f61bc0a3300 0x7f61bc0a57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61d4101d10 0x7f61d419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61d41026d0 0x7f61d41a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 --2- 172.21.15.28:0/3136924432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61d4100b10 0x7f61d419a070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 >> 172.21.15.28:0/3136924432 conn(0x7f61d40fc2a0 msgr2=0x7f61d4109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 shutdown_connections 2024-09-19T09:32:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:04.667+0000 7f61da5fd640 1 -- 172.21.15.28:0/3136924432 wait complete. 2024-09-19T09:32:04.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:04 smithi078 ceph-mon[35619]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:04.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:04 smithi038 ceph-mon[35271]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-09-19T09:32:05.326 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T09:32:05.588 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:05.799 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:05 smithi028 ceph-mon[30972]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:05.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:05 smithi078 ceph-mon[35619]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:05.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:05 smithi038 ceph-mon[35271]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:06.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 -- 172.21.15.28:0/2056097261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa784100b10 msgr2=0x7fa784100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:06.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 --2- 172.21.15.28:0/2056097261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa784100b10 0x7fa784100f10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa774009900 tx=0x7fa77402a410 comp rx=0 tx=0).stop 2024-09-19T09:32:06.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 -- 172.21.15.28:0/2056097261 shutdown_connections 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 --2- 172.21.15.28:0/2056097261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 0x7fa784102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 --2- 172.21.15.28:0/2056097261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7841026d0 0x7fa784108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 --2- 172.21.15.28:0/2056097261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa784100b10 0x7fa784100f10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 -- 172.21.15.28:0/2056097261 >> 172.21.15.28:0/2056097261 conn(0x7fa7840fc2a0 msgr2=0x7fa7840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 -- 172.21.15.28:0/2056097261 shutdown_connections 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.995+0000 7fa7891b9640 1 -- 172.21.15.28:0/2056097261 wait complete. 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.996+0000 7fa7891b9640 1 Processor -- start 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.996+0000 7fa7891b9640 1 -- start start 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa784100b10 0x7fa784197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 0x7fa784198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:06.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa78410c770 con 0x7fa7841026d0 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa78410c5f0 con 0x7fa784101d10 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa7891b9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa78410c8f0 con 0x7fa784100b10 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53170/0 (socket says 172.21.15.28:53170) 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 -- 172.21.15.28:0/1856128540 learned_addr learned my addr 172.21.15.28:0/1856128540 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa782d9d640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa784100b10 0x7fa784197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78259c640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 0x7fa784198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 -- 172.21.15.28:0/1856128540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa784100b10 msgr2=0x7fa784197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa784100b10 0x7fa784197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 -- 172.21.15.28:0/1856128540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 msgr2=0x7fa784198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 0x7fa784198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 -- 172.21.15.28:0/1856128540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78419fb40 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa78359e640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00eaa0 tx=0x7fa77c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00cdb0 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c005020 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.997+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0116b0 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.998+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78419fe30 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.998+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7841a0290 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:05.999+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7841a0550 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.000+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa77c005560 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.001+0000 7fa76bfff640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 0x7fa76c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.001+0000 7fa782d9d640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 0x7fa76c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.001+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa77c0cfcc0 con 0x7fa7841026d0 2024-09-19T09:32:06.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.001+0000 7fa782d9d640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 0x7fa76c0a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa774005ab0 tx=0x7fa774000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:06.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.006+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa77c016030 con 0x7fa7841026d0 2024-09-19T09:32:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.186+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa7841a0720 con 0x7fa76c0a3300 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.188+0000 7fa76bfff640 1 -- 172.21.15.28:0/1856128540 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7fa7841a0720 con 0x7fa76c0a3300 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:HOST ADDR LABELS STATUS 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028 172.21.15.28 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:smithi038 172.21.15.38 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:smithi078 172.21.15.78 2024-09-19T09:32:06.190 INFO:teuthology.orchestra.run.smithi028.stdout:3 hosts in cluster 2024-09-19T09:32:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 msgr2=0x7fa76c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 0x7fa76c0a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa774005ab0 tx=0x7fa774000bc0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 msgr2=0x7fa78419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00eaa0 tx=0x7fa77c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 shutdown_connections 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa76c0a3300 0x7fa76c0a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa784101d10 0x7fa784198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7841026d0 0x7fa78419f4d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 --2- 172.21.15.28:0/1856128540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa784100b10 0x7fa784197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 >> 172.21.15.28:0/1856128540 conn(0x7fa7840fc2a0 msgr2=0x7fa78406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 shutdown_connections 2024-09-19T09:32:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:06.191+0000 7fa7891b9640 1 -- 172.21.15.28:0/1856128540 wait complete. 2024-09-19T09:32:06.835 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:06 smithi028 ceph-mon[30972]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-19T09:32:06.835 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:06 smithi028 ceph-mon[30972]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:06 smithi078 ceph-mon[35619]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-19T09:32:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:06 smithi078 ceph-mon[35619]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:06.866 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T09:32:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:06 smithi038 ceph-mon[35271]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-19T09:32:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:06 smithi038 ceph-mon[35271]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:07.123 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 -- 172.21.15.28:0/888752357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 msgr2=0x7f432010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 --2- 172.21.15.28:0/888752357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f432010a100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4318009cd0 tx=0x7f431802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 -- 172.21.15.28:0/888752357 shutdown_connections 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 --2- 172.21.15.28:0/888752357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 0x7f4320103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 --2- 172.21.15.28:0/888752357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f432010a100 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 --2- 172.21.15.28:0/888752357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 0x7f43201002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 -- 172.21.15.28:0/888752357 >> 172.21.15.28:0/888752357 conn(0x7f43200fba40 msgr2=0x7f43200fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:07.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 -- 172.21.15.28:0/888752357 shutdown_connections 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.642+0000 7f432860a640 1 -- 172.21.15.28:0/888752357 wait complete. 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 Processor -- start 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 -- start start 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 0x7f4320102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 0x7f4320100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d890 con 0x7f4320103870 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d710 con 0x7f4320102eb0 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f432860a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f432010da10 con 0x7f43200ffed0 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f4326ba7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f4326ba7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53174/0 (socket says 172.21.15.28:53174) 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f4326ba7640 1 -- 172.21.15.28:0/3235242051 learned_addr learned my addr 172.21.15.28:0/3235242051 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:07.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f4325ba5640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 0x7f4320100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.643+0000 7f43263a6640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 0x7f4320102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 -- 172.21.15.28:0/3235242051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 msgr2=0x7f4320102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 0x7f4320102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 -- 172.21.15.28:0/3235242051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 msgr2=0x7f4320100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 0x7f4320100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 -- 172.21.15.28:0/3235242051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320101c60 con 0x7f4320103870 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f4326ba7640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4318009df0 tx=0x7f4318003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318004450 con 0x7f4320103870 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43180045f0 con 0x7f4320103870 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180467f0 con 0x7f4320103870 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43201a1fb0 con 0x7f4320103870 2024-09-19T09:32:07.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.644+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43201a2410 con 0x7f4320103870 2024-09-19T09:32:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.645+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4320109080 con 0x7f4320103870 2024-09-19T09:32:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.646+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f43180305a0 con 0x7f4320103870 2024-09-19T09:32:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.647+0000 7f43177fe640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 0x7f430c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.647+0000 7f43263a6640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 0x7f430c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.647+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f431802e030 con 0x7f4320103870 2024-09-19T09:32:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.647+0000 7f43263a6640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 0x7f430c0a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f432006a590 tx=0x7f431c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:07.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.650+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f431804b060 con 0x7f4320103870 2024-09-19T09:32:07.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.835+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4320062630 con 0x7f430c0a3350 2024-09-19T09:32:07.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.837+0000 7f43177fe640 1 -- 172.21.15.28:0/3235242051 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4320062630 con 0x7f430c0a3350 2024-09-19T09:32:07.839 INFO:teuthology.orchestra.run.smithi028.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T09:32:07.839 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747401F4480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T09:32:07.839 INFO:teuthology.orchestra.run.smithi028.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:32:07.839 INFO:teuthology.orchestra.run.smithi028.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:32:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 msgr2=0x7f430c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 0x7f430c0a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f432006a590 tx=0x7f431c00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 msgr2=0x7f43201013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4318009df0 tx=0x7f4318003d60 comp rx=0 tx=0).stop 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 shutdown_connections 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f430c0a3350 0x7f430c0a5810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4320102eb0 0x7f4320100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4320103870 0x7f43201013a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 --2- 172.21.15.28:0/3235242051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43200ffed0 0x7f4320102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 >> 172.21.15.28:0/3235242051 conn(0x7f43200fba40 msgr2=0x7f4320106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 shutdown_connections 2024-09-19T09:32:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:07.839+0000 7f432860a640 1 -- 172.21.15.28:0/3235242051 wait complete. 2024-09-19T09:32:08.462 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:32:08.474 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:32:08.474 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T09:32:08.731 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:08.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:08 smithi028 ceph-mon[30972]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:08.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:08 smithi028 ceph-mon[30972]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:08.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:08 smithi078 ceph-mon[35619]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:08.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:08 smithi078 ceph-mon[35619]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:08.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:08 smithi038 ceph-mon[35271]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:08.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:08 smithi038 ceph-mon[35271]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- 172.21.15.28:0/1437728745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0101cd0 msgr2=0x7fd7f0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1437728745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0101cd0 0x7fd7f0102150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009900 tx=0x7fd7e402a410 comp rx=0 tx=0).stop 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- 172.21.15.28:0/1437728745 shutdown_connections 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1437728745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0102690 0x7fd7f0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1437728745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0101cd0 0x7fd7f0102150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1437728745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0100ad0 0x7fd7f0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- 172.21.15.28:0/1437728745 >> 172.21.15.28:0/1437728745 conn(0x7fd7f00fc280 msgr2=0x7fd7f00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:09.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- 172.21.15.28:0/1437728745 shutdown_connections 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- 172.21.15.28:0/1437728745 wait complete. 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 Processor -- start 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.236+0000 7fd7f680e640 1 -- start start 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0100ad0 0x7fd7f0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0101cd0 0x7fd7f01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010c750 con 0x7fd7f0102690 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010c5d0 con 0x7fd7f0101cd0 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f680e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010c8d0 con 0x7fd7f0100ad0 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53188/0 (socket says 172.21.15.28:53188) 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 -- 172.21.15.28:0/1098586094 learned_addr learned my addr 172.21.15.28:0/1098586094 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7ef7fe640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0101cd0 0x7fd7f01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7effff640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0100ad0 0x7fd7f0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 -- 172.21.15.28:0/1098586094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0100ad0 msgr2=0x7fd7f0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0100ad0 0x7fd7f0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 -- 172.21.15.28:0/1098586094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0101cd0 msgr2=0x7fd7f01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0101cd0 0x7fd7f01984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.237+0000 7fd7f4dab640 1 -- 172.21.15.28:0/1098586094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f019fb90 con 0x7fd7f0102690 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7f4dab640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800eb10 tx=0x7fd7e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8004b20 con 0x7fd7f0102690 2024-09-19T09:32:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7e8004cc0 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f019fe80 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8010760 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.238+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a0330 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.239+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f01a05c0 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.240+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd7e8004e70 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.241+0000 7fd7ed7fa640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 0x7fd7d80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.241+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd7e80d03d0 con 0x7fd7f0102690 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.241+0000 7fd7effff640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 0x7fd7d80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.241+0000 7fd7effff640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 0x7fd7d80a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0101b30 tx=0x7fd7e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:09.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.244+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd7e809cfc0 con 0x7fd7f0102690 2024-09-19T09:32:09.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.458+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd7f01a0790 con 0x7fd7f0102690 2024-09-19T09:32:09.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.458+0000 7fd7ed7fa640 1 -- 172.21.15.28:0/1098586094 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3113 (secure 0 0 0) 0x7fd7e809c8c0 con 0x7fd7f0102690 2024-09-19T09:32:09.460 INFO:teuthology.orchestra.run.smithi028.stdout:e26 2024-09-19T09:32:09.460 INFO:teuthology.orchestra.run.smithi028.stdout:btime 2024-09-19T09:30:54:185960+0000 2024-09-19T09:32:09.460 INFO:teuthology.orchestra.run.smithi028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:legacy client fscid: 1 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:Filesystem 'cephfs' (1) 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:fs_name cephfs 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 26 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-09-19T09:30:43.054690+0000 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-09-19T09:30:54.185943+0000 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:tableserver 0 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:root 0 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:session_timeout 300 2024-09-19T09:32:09.461 INFO:teuthology.orchestra.run.smithi028.stdout:session_autoclose 300 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:max_file_size 1099511627776 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:max_xattr_size 65536 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:required_client_features {} 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure 0 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:last_failure_osd_epoch 0 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:max_mds 5 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:in 0,1,2,3,4 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:up {0=24499,1=24482,2=24478,3=24472,4=24455} 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:failed 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:damaged 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:stopped 2024-09-19T09:32:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:data_pools [3] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:metadata_pool 2 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:inline_data disabled 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:balancer 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:bal_rank_mask -1 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:standby_count_wanted 0 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:qdb_cluster leader: 24499 members: 24499,24482,24478,24472,24455 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.i{0:24499} state up:active seq 6 addr [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.l{0:24436} state up:standby-replay seq 1 addr [v2:172.21.15.78:6832/2992347158,v1:172.21.15.78:6833/2992347158] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b{1:24482} state up:active seq 6 addr [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.e{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.38:6834/347872214,v1:172.21.15.38:6835/347872214] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c{2:24478} state up:active seq 9 addr [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.k{2:24401} state up:standby-replay seq 1 addr [v2:172.21.15.38:6832/3013823435,v1:172.21.15.38:6833/3013823435] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.463 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.f{3:24472} state up:active seq 10 addr [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a{3:14628} state up:standby-replay seq 1 addr [v2:172.21.15.28:6840/438294996,v1:172.21.15.28:6841/438294996] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.h{4:24455} state up:active seq 9 addr [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.28:6838/1981224443,v1:172.21.15.28:6839/1981224443] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:Standby daemons: 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.28:6834/4241327209,v1:172.21.15.28:6835/4241327209] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.464 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.28:6836/558171377,v1:172.21.15.28:6837/558171377] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 msgr2=0x7fd7d80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 0x7fd7d80a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0101b30 tx=0x7fd7e000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 msgr2=0x7fd7f019f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800eb10 tx=0x7fd7e800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 shutdown_connections 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7d80a3300 0x7fd7d80a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0101cd0 0x7fd7f01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f0102690 0x7fd7f019f520 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 --2- 172.21.15.28:0/1098586094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7f0100ad0 0x7fd7f0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:09.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 >> 172.21.15.28:0/1098586094 conn(0x7fd7f00fc280 msgr2=0x7fd7f006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:09.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 shutdown_connections 2024-09-19T09:32:09.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:09.461+0000 7fd7f680e640 1 -- 172.21.15.28:0/1098586094 wait complete. 2024-09-19T09:32:09.758 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:09 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1098586094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:32:09.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:09 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1098586094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:32:09.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:09 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1098586094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:32:10.045 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T09:32:10.301 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:10.679 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:10 smithi028 ceph-mon[30972]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/2109052748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98104fd0 msgr2=0x7f7c981073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/2109052748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98104fd0 0x7f7c981073c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009900 tx=0x7f7c8c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/2109052748 shutdown_connections 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/2109052748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98107900 0x7f7c98109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/2109052748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98104fd0 0x7f7c981073c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/2109052748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 0x7f7c98104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/2109052748 >> 172.21.15.28:0/2109052748 conn(0x7f7c980fc2a0 msgr2=0x7f7c980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/2109052748 shutdown_connections 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.760+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/2109052748 wait complete. 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 Processor -- start 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 -- start start 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 0x7f7c98197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98104fd0 0x7f7c981984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9810d900 con 0x7f7c98107900 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9810d780 con 0x7f7c98104fd0 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c9d8ec640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9810da80 con 0x7f7c981026a0 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:10.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53202/0 (socket says 172.21.15.28:53202) 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.761+0000 7f7c977fe640 1 -- 172.21.15.28:0/4106083774 learned_addr learned my addr 172.21.15.28:0/4106083774 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c967fc640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98104fd0 0x7f7c981984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c96ffd640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 0x7f7c98197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 -- 172.21.15.28:0/4106083774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 msgr2=0x7f7c98197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 0x7f7c98197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 -- 172.21.15.28:0/4106083774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98104fd0 msgr2=0x7f7c981984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98104fd0 0x7f7c981984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 -- 172.21.15.28:0/4106083774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9819fbb0 con 0x7f7c98107900 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c977fe640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cce0 tx=0x7f7c9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000f450 con 0x7f7c98107900 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9819fea0 con 0x7f7c98107900 2024-09-19T09:32:10.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9000fad0 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90018b60 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.762+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c981a03d0 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.763+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c980fdd90 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.765+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7c9000fc80 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.765+0000 7f7c9c8ea640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 0x7f7c800a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.765+0000 7f7c96ffd640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 0x7f7c800a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.765+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7c900d02a0 con 0x7f7c98107900 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.766+0000 7f7c96ffd640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 0x7f7c800a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009990 tx=0x7f7c88009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:10.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.769+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c90014030 con 0x7f7c98107900 2024-09-19T09:32:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:10 smithi078 ceph-mon[35619]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:32:10.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:10 smithi038 ceph-mon[35271]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T09:32:10.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.946+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7c9802d7c0 con 0x7f7c98107900 2024-09-19T09:32:10.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.947+0000 7f7c9c8ea640 1 -- 172.21.15.28:0/4106083774 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5224 (secure 0 0 0) 0x7f7c9009ce90 con 0x7f7c98107900 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 82 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-09-19T09:22:57.723753+0000 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-09-19T09:31:49.348824+0000 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:crush_version 26 2024-09-19T09:32:10.949 INFO:teuthology.orchestra.run.smithi028.stdout:full_ratio 0.95 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:backfillfull_ratio 0.9 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:nearfull_ratio 0.85 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:require_min_compat_client luminous 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:min_compat_client luminous 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:require_osd_release squid 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:stretch_mode_enabled false 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:max_osd 12 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6802/1911211878,v1:172.21.15.28:6803/1911211878] [v2:172.21.15.28:6804/1911211878,v1:172.21.15.28:6805/1911211878] exists,up 8eb04994-c2e4-4428-8ec0-0a9597ab9498 2024-09-19T09:32:10.950 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6800/3199007949,v1:172.21.15.38:6801/3199007949] [v2:172.21.15.38:6802/3199007949,v1:172.21.15.38:6803/3199007949] exists,up c1e8a2f4-f1cf-48d0-af7d-f0e552676eec 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/720593762,v1:172.21.15.78:6801/720593762] [v2:172.21.15.78:6802/720593762,v1:172.21.15.78:6803/720593762] exists,up ae120b16-9641-4ffb-96b8-14da465baef8 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6810/3620512995,v1:172.21.15.28:6811/3620512995] [v2:172.21.15.28:6812/3620512995,v1:172.21.15.28:6813/3620512995] exists,up 225d1570-ea3d-414c-93b3-f440f93bf3f3 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 up in weight 1 up_from 32 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6808/465897146,v1:172.21.15.38:6809/465897146] [v2:172.21.15.38:6810/465897146,v1:172.21.15.38:6811/465897146] exists,up 7c9ef00f-d335-4c90-8ad3-541a898ddb60 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/3861457216,v1:172.21.15.78:6809/3861457216] [v2:172.21.15.78:6810/3861457216,v1:172.21.15.78:6811/3861457216] exists,up 05e37b87-069a-444c-9428-a4f59d08116d 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6 up in weight 1 up_from 44 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6818/2855006041,v1:172.21.15.28:6819/2855006041] [v2:172.21.15.28:6820/2855006041,v1:172.21.15.28:6821/2855006041] exists,up d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6816/917618380,v1:172.21.15.38:6817/917618380] [v2:172.21.15.38:6818/917618380,v1:172.21.15.38:6819/917618380] exists,up e19962bf-fa07-4593-96c4-8f8572d156dd 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8 up in weight 1 up_from 56 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/1616708891,v1:172.21.15.78:6817/1616708891] [v2:172.21.15.78:6818/1616708891,v1:172.21.15.78:6819/1616708891] exists,up 313b7ee6-931b-4eae-afa0-1f0950c1a0db 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9 up in weight 1 up_from 63 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6826/1812921948,v1:172.21.15.28:6827/1812921948] [v2:172.21.15.28:6828/1812921948,v1:172.21.15.28:6829/1812921948] exists,up a6ecdd64-1d04-4526-bf82-921f29aecf1f 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10 up in weight 1 up_from 67 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6824/1524737905,v1:172.21.15.38:6825/1524737905] [v2:172.21.15.38:6826/1524737905,v1:172.21.15.38:6827/1524737905] exists,up 20cc99ff-e67d-4b7a-b4da-e54e634584e6 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/2321429454,v1:172.21.15.78:6825/2321429454] [v2:172.21.15.78:6826/2321429454,v1:172.21.15.78:6827/2321429454] exists,up 3515d175-e41e-41d0-80d3-bc30d0657a87 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 2.5 [8,11] 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 2.8 [2,11] 2024-09-19T09:32:10.951 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 2.b [2,4] 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 3.c [3,9] 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 3.14 [4,6] 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 3.25 [5,9] 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:pg_upmap_items 3.36 [4,9] 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/4138562507 expires 2024-09-20T09:23:47.303309+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:6801/3407590911 expires 2024-09-20T09:23:47.303309+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/78423353 expires 2024-09-20T09:23:47.303309+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/1486296346 expires 2024-09-20T09:23:47.303309+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:6801/1042365655 expires 2024-09-20T09:23:22.243952+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:6800/1042365655 expires 2024-09-20T09:23:22.243952+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:6800/3407590911 expires 2024-09-20T09:23:47.303309+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/2768039084 expires 2024-09-20T09:23:22.243952+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/3190205246 expires 2024-09-20T09:23:22.243952+0000 2024-09-19T09:32:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:blocklist 172.21.15.28:0/2766061860 expires 2024-09-20T09:23:22.243952+0000 2024-09-19T09:32:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 msgr2=0x7f7c800a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 0x7f7c800a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009990 tx=0x7f7c88009110 comp rx=0 tx=0).stop 2024-09-19T09:32:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 msgr2=0x7f7c9819f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cce0 tx=0x7f7c9000a620 comp rx=0 tx=0).stop 2024-09-19T09:32:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 shutdown_connections 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c800a3300 0x7f7c800a57c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c98104fd0 0x7f7c981984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c98107900 0x7f7c9819f540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 --2- 172.21.15.28:0/4106083774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981026a0 0x7f7c98197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 >> 172.21.15.28:0/4106083774 conn(0x7f7c980fc2a0 msgr2=0x7f7c98102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 shutdown_connections 2024-09-19T09:32:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:10.949+0000 7f7c9d8ec640 1 -- 172.21.15.28:0/4106083774 wait complete. 2024-09-19T09:32:11.615 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T09:32:11.627 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:11.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:11 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4106083774' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:32:11.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:11 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4106083774' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:32:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:11.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:11 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4106083774' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 -- 172.21.15.28:0/3185729019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c102790 msgr2=0x7fd51c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3185729019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c102790 0x7fd51c1067e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd510009900 tx=0x7fd51002a410 comp rx=0 tx=0).stop 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 -- 172.21.15.28:0/3185729019 shutdown_connections 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3185729019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c106f10 0x7fd51c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3185729019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c102790 0x7fd51c1067e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3185729019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 -- 172.21.15.28:0/3185729019 >> 172.21.15.28:0/3185729019 conn(0x7fd51c0fd8c0 msgr2=0x7fd51c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 -- 172.21.15.28:0/3185729019 shutdown_connections 2024-09-19T09:32:12.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.341+0000 7fd522a3e640 1 -- 172.21.15.28:0/3185729019 wait complete. 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 Processor -- start 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 -- start start 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c102790 0x7fd51c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111f90 con 0x7fd51c106f10 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111e10 con 0x7fd51c101de0 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd522a3e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c112110 con 0x7fd51c102790 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd520fdb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c102790 0x7fd51c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd520fdb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53220/0 (socket says 172.21.15.28:53220) 2024-09-19T09:32:12.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.342+0000 7fd520fdb640 1 -- 172.21.15.28:0/3780211742 learned_addr learned my addr 172.21.15.28:0/3780211742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd51bfff640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 -- 172.21.15.28:0/3780211742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c102790 msgr2=0x7fd51c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c102790 0x7fd51c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 -- 172.21.15.28:0/3780211742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 msgr2=0x7fd51c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 -- 172.21.15.28:0/3780211742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c19d420 con 0x7fd51c106f10 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd51bfff640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd520fdb640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd51400b970 tx=0x7fd51400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514019070 con 0x7fd51c106f10 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c19d710 con 0x7fd51c106f10 2024-09-19T09:32:12.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51400cb40 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514007650 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.343+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c1a4ff0 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.344+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c1021c0 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.345+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd51401c070 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.346+0000 7fd5197fa640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 0x7fd5040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.346+0000 7fd51bfff640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 0x7fd5040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.346+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd5140cefa0 con 0x7fd51c106f10 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.347+0000 7fd51bfff640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 0x7fd5040a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005fd0 tx=0x7fd50c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.350+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd51409b5f0 con 0x7fd51c106f10 2024-09-19T09:32:12.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.564+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd51c10a150 con 0x7fd51c106f10 2024-09-19T09:32:12.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.565+0000 7fd5197fa640 1 -- 172.21.15.28:0/3780211742 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7fd51409f8f0 con 0x7fd51c106f10 2024-09-19T09:32:12.567 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:12.567 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:32:12.567 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:32:12.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 msgr2=0x7fd5040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:12.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 0x7fd5040a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005fd0 tx=0x7fd50c00b5f0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 msgr2=0x7fd51c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd51400b970 tx=0x7fd51400be30 comp rx=0 tx=0).stop 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 shutdown_connections 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5040a32d0 0x7fd5040a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd51c106f10 0x7fd51c19cd40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c102790 0x7fd51c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 --2- 172.21.15.28:0/3780211742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd51c101de0 0x7fd51c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.567+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 >> 172.21.15.28:0/3780211742 conn(0x7fd51c0fd8c0 msgr2=0x7fd51c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.568+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 shutdown_connections 2024-09-19T09:32:12.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:12.568+0000 7fd522a3e640 1 -- 172.21.15.28:0/3780211742 wait complete. 2024-09-19T09:32:12.849 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:12 smithi028 ceph-mon[30972]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:12.849 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:12 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3780211742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:12.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:12 smithi078 ceph-mon[35619]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:12 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3780211742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:12 smithi038 ceph-mon[35271]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:32:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:12 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3780211742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:13.173 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 -- 172.21.15.28:0/1742525952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c106f10 msgr2=0x7fe78c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 --2- 172.21.15.28:0/1742525952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c106f10 0x7fe78c10ea90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ca0 tx=0x7fe78402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 -- 172.21.15.28:0/1742525952 shutdown_connections 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 --2- 172.21.15.28:0/1742525952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c106f10 0x7fe78c10ea90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 --2- 172.21.15.28:0/1742525952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 0x7fe78c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 --2- 172.21.15.28:0/1742525952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c101de0 0x7fe78c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 -- 172.21.15.28:0/1742525952 >> 172.21.15.28:0/1742525952 conn(0x7fe78c0fd8c0 msgr2=0x7fe78c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 -- 172.21.15.28:0/1742525952 shutdown_connections 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.883+0000 7fe79401d640 1 -- 172.21.15.28:0/1742525952 wait complete. 2024-09-19T09:32:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 Processor -- start 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 -- start start 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 0x7fe78c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c106f10 0x7fe78c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c111f70 con 0x7fe78c101de0 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c111df0 con 0x7fe78c106f10 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe79401d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1120f0 con 0x7fe78c102790 2024-09-19T09:32:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53656/0 (socket says 172.21.15.28:53656) 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 -- 172.21.15.28:0/1751686293 learned_addr learned my addr 172.21.15.28:0/1751686293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe7915b8640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 0x7fe78c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe7925ba640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c106f10 0x7fe78c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 -- 172.21.15.28:0/1751686293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 msgr2=0x7fe78c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 0x7fe78c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 -- 172.21.15.28:0/1751686293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c106f10 msgr2=0x7fe78c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.884+0000 7fe791db9640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c106f10 0x7fe78c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe791db9640 1 -- 172.21.15.28:0/1751686293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c19d2e0 con 0x7fe78c101de0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe791db9640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c920 tx=0x7fe78800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788007c70 con 0x7fe78c101de0 2024-09-19T09:32:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c19d5d0 con 0x7fe78c101de0 2024-09-19T09:32:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe788007e10 con 0x7fe78c101de0 2024-09-19T09:32:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1a4eb0 con 0x7fe78c101de0 2024-09-19T09:32:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.885+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788018890 con 0x7fe78c101de0 2024-09-19T09:32:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.886+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c1021c0 con 0x7fe78c101de0 2024-09-19T09:32:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.887+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe788018a30 con 0x7fe78c101de0 2024-09-19T09:32:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.888+0000 7fe782ffd640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 0x7fe7780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.888+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe7880cfdc0 con 0x7fe78c101de0 2024-09-19T09:32:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.888+0000 7fe7915b8640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 0x7fe7780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.889+0000 7fe7915b8640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 0x7fe7780a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe78c108910 tx=0x7fe77c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:13.891+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe788014030 con 0x7fe78c101de0 2024-09-19T09:32:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.102+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe78c10a120 con 0x7fe78c101de0 2024-09-19T09:32:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.103+0000 7fe782ffd640 1 -- 172.21.15.28:0/1751686293 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11951 (secure 0 0 0) 0x7fe78809ca40 con 0x7fe78c101de0 2024-09-19T09:32:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:14.105 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"btime":"2024-09-19T09:30:54:185960+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:30:54.185943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:32:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 msgr2=0x7fe7780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 0x7fe7780a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe78c108910 tx=0x7fe77c00b390 comp rx=0 tx=0).stop 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 msgr2=0x7fe78c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c920 tx=0x7fe78800cde0 comp rx=0 tx=0).stop 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 shutdown_connections 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe7780a32d0 0x7fe7780a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe78c106f10 0x7fe78c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe78c102790 0x7fe78c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 --2- 172.21.15.28:0/1751686293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c101de0 0x7fe78c198330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 >> 172.21.15.28:0/1751686293 conn(0x7fe78c0fd8c0 msgr2=0x7fe78c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 shutdown_connections 2024-09-19T09:32:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:14.105+0000 7fe79401d640 1 -- 172.21.15.28:0/1751686293 wait complete. 2024-09-19T09:32:14.733 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:14 smithi028 ceph-mon[30972]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:32:14.733 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1751686293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:14.768 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:32:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:14 smithi078 ceph-mon[35619]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:32:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1751686293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:14.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:14 smithi038 ceph-mon[35271]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T09:32:14.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1751686293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:15.029 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:15.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 -- 172.21.15.28:0/3066132759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 msgr2=0x7ff5881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:15.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 --2- 172.21.15.28:0/3066132759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff5881067e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009900 tx=0x7ff57c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 -- 172.21.15.28:0/3066132759 shutdown_connections 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 --2- 172.21.15.28:0/3066132759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 0x7ff58810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 --2- 172.21.15.28:0/3066132759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff5881067e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 --2- 172.21.15.28:0/3066132759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 0x7ff5881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 -- 172.21.15.28:0/3066132759 >> 172.21.15.28:0/3066132759 conn(0x7ff5880fd8c0 msgr2=0x7ff5880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 -- 172.21.15.28:0/3066132759 shutdown_connections 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.493+0000 7ff58e07d640 1 -- 172.21.15.28:0/3066132759 wait complete. 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 Processor -- start 2024-09-19T09:32:15.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 -- start start 2024-09-19T09:32:15.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 0x7ff5880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:15.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 0x7ff588074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff588111f30 con 0x7ff588102790 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff588111db0 con 0x7ff588101de0 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff58e07d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881120b0 con 0x7ff588106f10 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53666/0 (socket says 172.21.15.28:53666) 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 -- 172.21.15.28:0/4273761326 learned_addr learned my addr 172.21.15.28:0/4273761326 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff5877fe640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 0x7ff5880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff587fff640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 0x7ff588074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 -- 172.21.15.28:0/4273761326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 msgr2=0x7ff588074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 0x7ff588074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.494+0000 7ff586ffd640 1 -- 172.21.15.28:0/4273761326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 msgr2=0x7ff5880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff586ffd640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 0x7ff5880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff586ffd640 1 -- 172.21.15.28:0/4273761326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5881a68f0 con 0x7ff588102790 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff586ffd640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006010 tx=0x7ff57c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c03f030 con 0x7ff588102790 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c003ce0 con 0x7ff588102790 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5881a6b80 con 0x7ff588102790 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c035070 con 0x7ff588102790 2024-09-19T09:32:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.495+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a7060 con 0x7ff588102790 2024-09-19T09:32:15.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.496+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5881021c0 con 0x7ff588102790 2024-09-19T09:32:15.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.497+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff57c042030 con 0x7ff588102790 2024-09-19T09:32:15.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.498+0000 7ff584ff9640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 0x7ff5700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:15.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.498+0000 7ff5877fe640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 0x7ff5700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:15.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.499+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff57c0f0280 con 0x7ff588102790 2024-09-19T09:32:15.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.499+0000 7ff5877fe640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 0x7ff5700a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff57800bbe0 tx=0x7ff57800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.502+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff57c0bc880 con 0x7ff588102790 2024-09-19T09:32:15.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.677+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff58810a0f0 con 0x7ff588102790 2024-09-19T09:32:15.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.679+0000 7ff584ff9640 1 -- 172.21.15.28:0/4273761326 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22580 (secure 0 0 0) 0x7ff57c0c0b80 con 0x7ff588102790 2024-09-19T09:32:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:15.681 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:31:49.348824+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 msgr2=0x7ff5700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 0x7ff5700a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff57800bbe0 tx=0x7ff57800b390 comp rx=0 tx=0).stop 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 msgr2=0x7ff588074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006010 tx=0x7ff57c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 shutdown_connections 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5700a32d0 0x7ff5700a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588106f10 0x7ff588074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff588102790 0x7ff588074a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 --2- 172.21.15.28:0/4273761326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff588101de0 0x7ff5880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:15.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 >> 172.21.15.28:0/4273761326 conn(0x7ff5880fd8c0 msgr2=0x7ff5880fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:15.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 shutdown_connections 2024-09-19T09:32:15.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:15.681+0000 7ff58e07d640 1 -- 172.21.15.28:0/4273761326 wait complete. 2024-09-19T09:32:16.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:30:43.054690+0000', 'modified': '2024-09-19T09:30:54.185943+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24499, 'mds_1': 24482, 'mds_2': 24478, 'mds_3': 24472, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6839/1981224443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6838', 'nonce': 1981224443}, {'type': 'v1', 'addr': '172.21.15.28:6839', 'nonce': 1981224443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6841/438294996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6840', 'nonce': 438294996}, {'type': 'v1', 'addr': '172.21.15.28:6841', 'nonce': 438294996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3013823435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3013823435}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3013823435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/347872214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 347872214}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 347872214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6833/2992347158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 2992347158}, {'type': 'v1', 'addr': '172.21.15.78:6833', 'nonce': 2992347158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.38:6837/560457008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 560457008}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 560457008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.78:6835/2086564778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 2086564778}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 2086564778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.78:6837/2648819818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2648819818}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2648819818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.38:6839/3114720816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 3114720816}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 3114720816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.78:6839/3581897338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3581897338}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3581897338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24499, 24482, 24478, 24472, 24455]} 2024-09-19T09:32:16.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:32:16.291 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:32:16.549 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:16 smithi078 ceph-mon[35619]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-19T09:32:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:16 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4273761326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:16.904 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:16 smithi028 ceph-mon[30972]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-19T09:32:16.905 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:16 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4273761326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:16 smithi038 ceph-mon[35271]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-19T09:32:16.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:16 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4273761326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 -- 172.21.15.28:0/2051458426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec102d40 msgr2=0x7fa2ec103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 --2- 172.21.15.28:0/2051458426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec102d40 0x7fa2ec103120 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009900 tx=0x7fa2dc02a480 comp rx=0 tx=0).stop 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 -- 172.21.15.28:0/2051458426 shutdown_connections 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 --2- 172.21.15.28:0/2051458426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec10c170 0x7fa2ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 --2- 172.21.15.28:0/2051458426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 --2- 172.21.15.28:0/2051458426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec102d40 0x7fa2ec103120 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 -- 172.21.15.28:0/2051458426 >> 172.21.15.28:0/2051458426 conn(0x7fa2ec0fc7a0 msgr2=0x7fa2ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 -- 172.21.15.28:0/2051458426 shutdown_connections 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.987+0000 7fa2f238c640 1 -- 172.21.15.28:0/2051458426 wait complete. 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 Processor -- start 2024-09-19T09:32:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 -- start start 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec102d40 0x7fa2ec1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111f70 con 0x7fa2ec10c170 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec111df0 con 0x7fa2ec102d40 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f238c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1120f0 con 0x7fa2ec103660 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f0929640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f0929640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53672/0 (socket says 172.21.15.28:53672) 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.988+0000 7fa2f0929640 1 -- 172.21.15.28:0/1747306532 learned_addr learned my addr 172.21.15.28:0/1747306532 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec102d40 0x7fa2ec1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:16.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 -- 172.21.15.28:0/1747306532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 msgr2=0x7fa2ec100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 -- 172.21.15.28:0/1747306532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec102d40 msgr2=0x7fa2ec1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec102d40 0x7fa2ec1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 -- 172.21.15.28:0/1747306532 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec1a68c0 con 0x7fa2ec10c170 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f0929640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400cce0 tx=0x7fa2e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2e97fa640 1 -- 172.21.15.28:0/1747306532 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400f450 con 0x7fa2ec10c170 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2e97fa640 1 -- 172.21.15.28:0/1747306532 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e400fad0 con 0x7fa2ec10c170 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec1a6bb0 con 0x7fa2ec10c170 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2e97fa640 1 -- 172.21.15.28:0/1747306532 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4018c10 con 0x7fa2ec10c170 2024-09-19T09:32:16.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.989+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a70a0 con 0x7fa2ec10c170 2024-09-19T09:32:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2e97fa640 1 -- 172.21.15.28:0/1747306532 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa2e400f5f0 con 0x7fa2ec10c170 2024-09-19T09:32:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 Processor -- start 2024-09-19T09:32:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 -- start start 2024-09-19T09:32:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x56186199bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199c2f0 0x5618619a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561861902090 con 0x5618619a6bb0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618618a9a70 con 0x56186199b9d0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2f4765740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618618fe050 con 0x56186199c2f0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.992+0000 7fa2eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53676/0 (socket says 172.21.15.28:53676) 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/4275340947 learned_addr learned my addr 172.21.15.28:0/4275340947 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199c2f0 0x5618619a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x56186199bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2e97fa640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 0x7fa2d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2f0929640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 0x7fa2d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/4275340947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199c2f0 msgr2=0x5618619a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199c2f0 0x5618619a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/4275340947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 msgr2=0x56186199bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x56186199bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/4275340947 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618618b6130 con 0x5618619a6bb0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2e97fa640 1 -- 172.21.15.28:0/1747306532 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa2e40d01c0 con 0x7fa2ec10c170 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0013940 tx=0x7fa2e000b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90077a3775b619d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2d27fc640 1 -- 172.21.15.28:0/4275340947 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0004020 con 0x5618619a6bb0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2d27fc640 1 -- 172.21.15.28:0/4275340947 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e000ee40 con 0x5618619a6bb0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2f0929640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 0x7fa2d40a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2e406b3c0 tx=0x7fa2e406f530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.993+0000 7fa2d27fc640 1 -- 172.21.15.28:0/4275340947 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0003910 con 0x5618619a6bb0 2024-09-19T09:32:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- 172.21.15.28:0/4275340947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 msgr2=0x5618619a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0013940 tx=0x7fa2e000b520 comp rx=0 tx=0).stop 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- 172.21.15.28:0/4275340947 shutdown_connections 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618619a6bb0 0x5618619a8fa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199c2f0 0x5618619a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 --2- 172.21.15.28:0/4275340947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x56186199bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- 172.21.15.28:0/4275340947 >> 172.21.15.28:0/4275340947 conn(0x561861997f60 msgr2=0x561861998340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- 172.21.15.28:0/4275340947 shutdown_connections 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- 172.21.15.28:0/4275340947 wait complete. 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.994+0000 7fa2f4765740 1 -- start start 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 Processor -- start 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 0x5618619a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618619a9780 con 0x56186199b9d0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618618bcf20 con 0x56186199c2f0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618619a9900 con 0x5618619a6bb0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2f4765740 10 client.0 _ll_register_callbacks cb 0x561861886030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53680/0 (socket says 172.21.15.28:53680) 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/3568835047 learned_addr learned my addr 172.21.15.28:0/3568835047 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 0x5618619a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/3568835047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 msgr2=0x5618619a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 msgr2=0x5618619a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 0x5618619a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2eb7fe640 1 -- 172.21.15.28:0/3568835047 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618616732b0 con 0x56186199b9d0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.995+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 0x5618619a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00058c0 tx=0x7fa2e0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319431cd2a530269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000e960 con 0x56186199b9d0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa2e000eb00 con 0x56186199b9d0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e000ee40 con 0x56186199b9d0 2024-09-19T09:32:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2e0030050 con 0x56186199b9d0 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2f4765740 10 client.15138 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618619b5060 con 0x56186199b9d0 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2e000ee40 con 0x56186199b9d0 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa2e0011440 con 0x56186199b9d0 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2f4765740 10 client.15138 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.996+0000 7fa2f4765740 10 client.15138 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2f4765740 20 client.15138 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5618619b7370 0x5618619b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2f4765740 4 client.15138 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186190f600 con 0x5618619b7370 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5618619b7370 0x5618619b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2cb7fe640 10 client.15138 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.997+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5618619b7370 0x5618619b87e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e986fa759335df4 server_cookie=2d7d20462dafd831 in_seq=0 out_seq=0 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/3568835047 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56186190f600 con 0x5618619b7370 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2cb7fe640 10 client.15138 handle_command_reply: tid=0 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 client.15138 shutdown 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 msgr2=0x5618619a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00058c0 tx=0x7fa2e0005760 comp rx=0 tx=0).stop 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 shutdown_connections 2024-09-19T09:32:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5618619b7370 0x5618619b87e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199c2f0 0x5618619a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3568835047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199b9d0 0x5618619a63e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 >> 172.21.15.28:0/3568835047 conn(0x561861997f60 msgr2=0x56186199dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.998+0000 7fa2cb7fe640 0 client.15138 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.999+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 shutdown_connections 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.999+0000 7fa2f4765740 1 -- 172.21.15.28:0/3568835047 wait complete. 2024-09-19T09:32:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:16.999+0000 7fa2f4765740 20 client.15138 trim_cache size 0 max 16384 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 Processor -- start 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 -- start start 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x5618619b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618619b6b20 con 0x56186199c2f0 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618619b71f0 con 0x56186199b9d0 2024-09-19T09:32:17.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2f4765740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618619b78c0 con 0x5618619a6bb0 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53682/0 (socket says 172.21.15.28:53682) 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 -- 172.21.15.28:0/3279079256 learned_addr learned my addr 172.21.15.28:0/3279079256 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x5618619b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 -- 172.21.15.28:0/3279079256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 msgr2=0x5618619b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 -- 172.21.15.28:0/3279079256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 msgr2=0x5618619b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x5618619b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.017+0000 7fa2ebfff640 1 -- 172.21.15.28:0/3279079256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618619a4250 con 0x56186199c2f0 2024-09-19T09:32:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc02a9a0 tx=0x7fa2dc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acf471a114ea1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:17.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2d1ffb640 1 -- 172.21.15.28:0/3279079256 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc048070 con 0x56186199c2f0 2024-09-19T09:32:17.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2d1ffb640 1 -- 172.21.15.28:0/3279079256 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2dc005020 con 0x56186199c2f0 2024-09-19T09:32:17.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2d1ffb640 1 -- 172.21.15.28:0/3279079256 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc035040 con 0x56186199c2f0 2024-09-19T09:32:17.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 -- 172.21.15.28:0/3279079256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 msgr2=0x5618619b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc02a9a0 tx=0x7fa2dc005a20 comp rx=0 tx=0).stop 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 -- 172.21.15.28:0/3279079256 shutdown_connections 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618619a6bb0 0x5618619b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619b8110 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 --2- 172.21.15.28:0/3279079256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56186199b9d0 0x5618619b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.018+0000 7fa2f4765740 1 -- 172.21.15.28:0/3279079256 >> 172.21.15.28:0/3279079256 conn(0x561861997f60 msgr2=0x561861998340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- 172.21.15.28:0/3279079256 shutdown_connections 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- 172.21.15.28:0/3279079256 wait complete. 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- start start 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 Processor -- start 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199b9d0 0x5618619bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 0x561861a1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618619b8660 con 0x56186199c2f0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618619b7f90 con 0x5618619a6bb0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.019+0000 7fa2f4765740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618619b8d30 con 0x56186199b9d0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2f4765740 10 client.0 _ll_register_callbacks cb 0x7fa2e000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53686/0 (socket says 172.21.15.28:53686) 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 -- 172.21.15.28:0/331690302 learned_addr learned my addr 172.21.15.28:0/331690302 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199b9d0 0x5618619bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 0x561861a1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 -- 172.21.15.28:0/331690302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199b9d0 msgr2=0x5618619bbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199b9d0 0x5618619bbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 -- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 msgr2=0x561861a1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:17.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 0x561861a1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 -- 172.21.15.28:0/331690302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618619b6950 con 0x56186199c2f0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 0x561861a1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2ebfff640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc02a4c0 tx=0x7fa2dc004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60612f319779153 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc048070 con 0x56186199c2f0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa2dc004920 con 0x56186199c2f0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc005020 con 0x56186199c2f0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc05b020 con 0x56186199c2f0 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.020+0000 7fa2f4765740 10 client.15144 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:17.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561861a20f40 con 0x56186199c2f0 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2dc005020 con 0x56186199c2f0 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa2dc03b9d0 con 0x56186199c2f0 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 20 client.15144 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x561861a25630 0x561861a24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 4 client.15144 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "status"}) -- 0x561861909b10 con 0x561861a25630 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.021+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x561861a25630 0x561861a24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:17.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2eb7fe640 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x561861a25630 0x561861a24340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76475df0b5b53c3c server_cookie=b5e02eb04ab49845 in_seq=0 out_seq=0 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2cb7fe640 10 client.15144 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2cb7fe640 1 -- 172.21.15.28:0/331690302 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561861909b10 con 0x561861a25630 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2cb7fe640 10 client.15144 handle_command_reply: tid=0 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 client.15144 shutdown 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 msgr2=0x5618619bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc02a4c0 tx=0x7fa2dc004830 comp rx=0 tx=0).stop 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 shutdown_connections 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x561861a25630 0x561861a24340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618619a6bb0 0x561861a1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56186199c2f0 0x5618619bc090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 --2- 172.21.15.28:0/331690302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56186199b9d0 0x5618619bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 >> 172.21.15.28:0/331690302 conn(0x561861997f60 msgr2=0x5618619a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2cb7fe640 0 client.15144 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 shutdown_connections 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 1 -- 172.21.15.28:0/331690302 wait complete. 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.022+0000 7fa2f4765740 20 client.15144 trim_cache size 0 max 16384 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 4, 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24455, 2024-09-19T09:32:17.031 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 83.844697080000003, 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 114.735491565 2024-09-19T09:32:17.032 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 msgr2=0x7fa2d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 0x7fa2d40a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2e406b3c0 tx=0x7fa2e406f530 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 msgr2=0x7fa2ec101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400cce0 tx=0x7fa2e400a620 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 shutdown_connections 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa2d40a32d0 0x7fa2d40a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2ec10c170 0x7fa2ec101420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2ec103660 0x7fa2ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 --2- 172.21.15.28:0/1747306532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec102d40 0x7fa2ec1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:17.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 >> 172.21.15.28:0/1747306532 conn(0x7fa2ec0fc7a0 msgr2=0x7fa2ec0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:17.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 shutdown_connections 2024-09-19T09:32:17.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:17.023+0000 7fa2f238c640 1 -- 172.21.15.28:0/1747306532 wait complete. 2024-09-19T09:32:17.906 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:32:18.164 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:18.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2353124515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 msgr2=0x7fd29810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2353124515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd29810da40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd290009ca0 tx=0x7fd29002a950 comp rx=0 tx=0).stop 2024-09-19T09:32:18.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2353124515 shutdown_connections 2024-09-19T09:32:18.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2353124515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298102570 0x7fd2981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2353124515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd29810da40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2353124515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298100e20 0x7fd298101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2353124515 >> 172.21.15.28:0/2353124515 conn(0x7fd2980fc7d0 msgr2=0x7fd2980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2353124515 shutdown_connections 2024-09-19T09:32:18.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.622+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2353124515 wait complete. 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 Processor -- start 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 -- start start 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 0x7fd2981993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298102570 0x7fd298199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110f80 con 0x7fd298102ef0 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110e00 con 0x7fd298100e20 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd2a05c6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd298111100 con 0x7fd298102570 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd29eb63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd29eb63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53706/0 (socket says 172.21.15.28:53706) 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.623+0000 7fd29eb63640 1 -- 172.21.15.28:0/2216995648 learned_addr learned my addr 172.21.15.28:0/2216995648 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29db61640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298102570 0x7fd298199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29e362640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 0x7fd2981993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 -- 172.21.15.28:0/2216995648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298102570 msgr2=0x7fd298199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298102570 0x7fd298199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 -- 172.21.15.28:0/2216995648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 msgr2=0x7fd2981993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 0x7fd2981993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 -- 172.21.15.28:0/2216995648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2981a47d0 con 0x7fd298102ef0 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29e362640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 0x7fd2981993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd29eb63640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd29002ae70 tx=0x7fd29002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd28f7fe640 1 -- 172.21.15.28:0/2216995648 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290034040 con 0x7fd298102ef0 2024-09-19T09:32:18.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981a4a60 con 0x7fd298102ef0 2024-09-19T09:32:18.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd28f7fe640 1 -- 172.21.15.28:0/2216995648 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290030920 con 0x7fd298102ef0 2024-09-19T09:32:18.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd28f7fe640 1 -- 172.21.15.28:0/2216995648 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290045480 con 0x7fd298102ef0 2024-09-19T09:32:18.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.624+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a4f90 con 0x7fd298102ef0 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.627+0000 7fd28f7fe640 1 -- 172.21.15.28:0/2216995648 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd290037020 con 0x7fd298102ef0 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.627+0000 7fd2a299f740 1 Processor -- start 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 -- start start 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 0x55cae8b3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cae8a98090 con 0x55cae8b3cbb0 2024-09-19T09:32:18.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cae8a3fa70 con 0x55cae8b322f0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd2a299f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cae8a94050 con 0x55cae8b319d0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53718/0 (socket says 172.21.15.28:53718) 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 -- 172.21.15.28:0/3370956011 learned_addr learned my addr 172.21.15.28:0/3370956011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29e362640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 0x55cae8b3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd28f7fe640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 0x7fd2840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29eb63640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 0x7fd2840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 -- 172.21.15.28:0/3370956011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 msgr2=0x55cae8b31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 -- 172.21.15.28:0/3370956011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 msgr2=0x55cae8b3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.628+0000 7fd29db61640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 0x55cae8b3c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd29db61640 1 -- 172.21.15.28:0/3370956011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae8a4c130 con 0x55cae8b3cbb0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd29e362640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 0x55cae8b3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd28f7fe640 1 -- 172.21.15.28:0/2216995648 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd2900f07f0 con 0x7fd298102ef0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd29db61640 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd2880126a0 tx=0x7fd28800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edd1b6710f14a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd28cff9640 1 -- 172.21.15.28:0/3370956011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288004020 con 0x55cae8b3cbb0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd28cff9640 1 -- 172.21.15.28:0/3370956011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880041c0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd29eb63640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 0x7fd2840a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2900bcb40 tx=0x7fd290087c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.629+0000 7fd28cff9640 1 -- 172.21.15.28:0/3370956011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28801e850 con 0x55cae8b3cbb0 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- 172.21.15.28:0/3370956011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 msgr2=0x55cae8b3efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd2880126a0 tx=0x7fd28800b650 comp rx=0 tx=0).stop 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- 172.21.15.28:0/3370956011 shutdown_connections 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b322f0 0x55cae8b3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b3efa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 --2- 172.21.15.28:0/3370956011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- 172.21.15.28:0/3370956011 >> 172.21.15.28:0/3370956011 conn(0x55cae8b2df60 msgr2=0x55cae8b2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- 172.21.15.28:0/3370956011 shutdown_connections 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- 172.21.15.28:0/3370956011 wait complete. 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 -- start start 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.630+0000 7fd2a299f740 1 Processor -- start 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b3c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b322f0 0x55cae8b396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b3f780 con 0x55cae8b3cbb0 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cae8a52f20 con 0x55cae8b319d0 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b3f900 con 0x55cae8b322f0 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 10 client.0 _ll_register_callbacks cb 0x55cae8a1c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:18.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53730/0 (socket says 172.21.15.28:53730) 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 -- 172.21.15.28:0/993982444 learned_addr learned my addr 172.21.15.28:0/993982444 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b3c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29e362640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b322f0 0x55cae8b396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 -- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b322f0 msgr2=0x55cae8b396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b322f0 0x55cae8b396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 -- 172.21.15.28:0/993982444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 msgr2=0x55cae8b3c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b3c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 -- 172.21.15.28:0/993982444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae88092b0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd29db61640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd288003de0 tx=0x7fd288003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fa5b0ed8f9c41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd281ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28801e650 con 0x55cae8b3cbb0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd28801e7f0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.631+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2880041c0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd28801e9f0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 10 client.15156 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cae8b4b010 con 0x55cae8b3cbb0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2880041c0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd288027b80 con 0x55cae8b3cbb0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 10 client.15156 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 20 client.15156 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8b4d3a0 0x55cae8b4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 4 client.15156 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae8aa5600 con 0x55cae8b4d3a0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.632+0000 7fd29e362640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8b4d3a0 0x55cae8b4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.633+0000 7fd29e362640 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8b4d3a0 0x55cae8b4e810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c7bfd378e6bed98 server_cookie=70b5245345f4f67e in_seq=0 out_seq=0 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.633+0000 7fd281ffb640 10 client.15156 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.633+0000 7fd281ffb640 1 -- 172.21.15.28:0/993982444 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cae8aa5600 con 0x55cae8b4d3a0 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.633+0000 7fd281ffb640 10 client.15156 handle_command_reply: tid=0 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.633+0000 7fd2a299f740 1 client.15156 shutdown 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 msgr2=0x55cae8b39c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd288003de0 tx=0x7fd288003ee0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 shutdown_connections 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8b4d3a0 0x55cae8b4e810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b3cbb0 0x55cae8b39c20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b322f0 0x55cae8b396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 --2- 172.21.15.28:0/993982444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 >> 172.21.15.28:0/993982444 conn(0x55cae8b2df60 msgr2=0x55cae8b33c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd281ffb640 0 client.15156 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:18.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 shutdown_connections 2024-09-19T09:32:18.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 1 -- 172.21.15.28:0/993982444 wait complete. 2024-09-19T09:32:18.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.634+0000 7fd2a299f740 20 client.15156 trim_cache size 0 max 16384 2024-09-19T09:32:18.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 Processor -- start 2024-09-19T09:32:18.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 -- start start 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b4ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b3cbb0 0x55cae8b4c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4cb50 con 0x55cae8b322f0 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4d220 con 0x55cae8b319d0 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd2a299f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4d8f0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53740/0 (socket says 172.21.15.28:53740) 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 -- 172.21.15.28:0/2216111608 learned_addr learned my addr 172.21.15.28:0/2216111608 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29db61640 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b4ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 -- 172.21.15.28:0/2216111608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b3cbb0 msgr2=0x55cae8b4c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b3cbb0 0x55cae8b4c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 -- 172.21.15.28:0/2216111608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 msgr2=0x55cae8b4ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b4ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29e362640 1 -- 172.21.15.28:0/2216111608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae8b3c700 con 0x55cae8b322f0 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.652+0000 7fd29db61640 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b4ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd29e362640 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55cae8b2edc0 tx=0x7fd294013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09903d639344fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd283fff640 1 -- 172.21.15.28:0/2216111608 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29401c460 con 0x55cae8b322f0 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd283fff640 1 -- 172.21.15.28:0/2216111608 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29401cae0 con 0x55cae8b322f0 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd283fff640 1 -- 172.21.15.28:0/2216111608 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294011c00 con 0x55cae8b322f0 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 -- 172.21.15.28:0/2216111608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 msgr2=0x55cae8b4d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55cae8b2edc0 tx=0x7fd294013830 comp rx=0 tx=0).stop 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 -- 172.21.15.28:0/2216111608 shutdown_connections 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b3cbb0 0x55cae8b4c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b4d3a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2216111608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b319d0 0x55cae8b4ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 -- 172.21.15.28:0/2216111608 >> 172.21.15.28:0/2216111608 conn(0x55cae8b2df60 msgr2=0x55cae8b2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.653+0000 7fd2a299f740 1 -- 172.21.15.28:0/2216111608 shutdown_connections 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 -- 172.21.15.28:0/2216111608 wait complete. 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 -- start start 2024-09-19T09:32:18.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 Processor -- start 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b4ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b3cbb0 0x55cae8bb5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4e690 con 0x55cae8b322f0 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4dfc0 con 0x55cae8b3cbb0 2024-09-19T09:32:18.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cae8b4ed60 con 0x55cae8b319d0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd2a299f740 10 client.0 _ll_register_callbacks cb 0x7fd29400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd29e362640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd29e362640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53748/0 (socket says 172.21.15.28:53748) 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd29e362640 1 -- 172.21.15.28:0/2826469033 learned_addr learned my addr 172.21.15.28:0/2826469033 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd29db61640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b3cbb0 0x55cae8bb5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.654+0000 7fd29db61640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b4ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 -- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 msgr2=0x55cae8b4ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b4ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 -- 172.21.15.28:0/2826469033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b3cbb0 msgr2=0x55cae8bb5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b3cbb0 0x55cae8bb5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 -- 172.21.15.28:0/2826469033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae8bb5fe0 con 0x55cae8b322f0 2024-09-19T09:32:18.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29db61640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b4ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd29e362640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd29400be40 tx=0x7fd29400bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d718b8ccd3b9bf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd281ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29401c660 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd29401c800 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29401cae0 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd294036020 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd2a299f740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cae8bb7230 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.655+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29401cae0 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd2940046c0 con 0x55cae8b322f0 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:18.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 10 client.15162 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 20 client.15162 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8bbba50 0x55cae8bba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 4 client.15162 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "status"}) -- 0x55cae8a9fb10 con 0x55cae8bbba50 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd29db61640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8bbba50 0x55cae8bba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd29db61640 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8bbba50 0x55cae8bba760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5197ee5e5512770 server_cookie=8177efe34c6ed0f6 in_seq=0 out_seq=0 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.656+0000 7fd281ffb640 10 client.15162 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd281ffb640 1 -- 172.21.15.28:0/2826469033 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cae8a9fb10 con 0x55cae8bbba50 2024-09-19T09:32:18.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd281ffb640 10 client.15162 handle_command_reply: tid=0 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 client.15162 shutdown 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 msgr2=0x55cae8b50480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd29400be40 tx=0x7fd29400bb70 comp rx=0 tx=0).stop 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 shutdown_connections 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55cae8bbba50 0x55cae8bba760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cae8b3cbb0 0x55cae8bb5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cae8b322f0 0x55cae8b50480 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 --2- 172.21.15.28:0/2826469033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cae8b319d0 0x55cae8b4ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 >> 172.21.15.28:0/2826469033 conn(0x55cae8b2df60 msgr2=0x55cae8b4c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd281ffb640 0 client.15162 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 shutdown_connections 2024-09-19T09:32:18.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 1 -- 172.21.15.28:0/2826469033 wait complete. 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a299f740 20 client.15162 trim_cache size 0 max 16384 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 3, 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24472, 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:18.666 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 86.484440477000007, 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 121.318149328 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 msgr2=0x7fd2840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 0x7fd2840a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2900bcb40 tx=0x7fd290087c70 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 msgr2=0x7fd298199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd29002ae70 tx=0x7fd29002f5d0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 shutdown_connections 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd2840a3240 0x7fd2840a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd298102ef0 0x7fd298199e90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd298102570 0x7fd298199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.657+0000 7fd2a05c6640 1 --2- 172.21.15.28:0/2216995648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd298100e20 0x7fd2981993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.658+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 >> 172.21.15.28:0/2216995648 conn(0x7fd2980fc7d0 msgr2=0x7fd2981068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.658+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 shutdown_connections 2024-09-19T09:32:18.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:18.658+0000 7fd2a05c6640 1 -- 172.21.15.28:0/2216995648 wait complete. 2024-09-19T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:18 smithi078 ceph-mon[35619]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:18 smithi078 ceph-mon[35619]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:32:18.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:18 smithi038 ceph-mon[35271]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:32:18.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:18 smithi038 ceph-mon[35271]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:32:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:18 smithi028 ceph-mon[30972]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-19T09:32:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:18 smithi028 ceph-mon[30972]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:32:19.460 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:32:19.717 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:20.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 -- 172.21.15.28:0/2423234443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 msgr2=0x7fee78110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 --2- 172.21.15.28:0/2423234443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee78110cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fee70009900 tx=0x7fee7002a410 comp rx=0 tx=0).stop 2024-09-19T09:32:20.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 -- 172.21.15.28:0/2423234443 shutdown_connections 2024-09-19T09:32:20.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 --2- 172.21.15.28:0/2423234443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee781049c0 0x7fee78108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 --2- 172.21.15.28:0/2423234443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee78110cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 --2- 172.21.15.28:0/2423234443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee78104010 0x7fee781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 -- 172.21.15.28:0/2423234443 >> 172.21.15.28:0/2423234443 conn(0x7fee780ffa50 msgr2=0x7fee78101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 -- 172.21.15.28:0/2423234443 shutdown_connections 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.146+0000 7fee801c0640 1 -- 172.21.15.28:0/2423234443 wait complete. 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 Processor -- start 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 -- start start 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee78104010 0x7fee7819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee781049c0 0x7fee7819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee78114150 con 0x7fee78109140 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee78113fd0 con 0x7fee78104010 2024-09-19T09:32:20.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee801c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee781142d0 con 0x7fee781049c0 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.147+0000 7fee7e75d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53766/0 (socket says 172.21.15.28:53766) 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 -- 172.21.15.28:0/2221991868 learned_addr learned my addr 172.21.15.28:0/2221991868 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7d75b640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee781049c0 0x7fee7819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7df5c640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee78104010 0x7fee7819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 -- 172.21.15.28:0/2221991868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee781049c0 msgr2=0x7fee7819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee781049c0 0x7fee7819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 -- 172.21.15.28:0/2221991868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee78104010 msgr2=0x7fee7819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee78104010 0x7fee7819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 -- 172.21.15.28:0/2221991868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7819f5a0 con 0x7fee78109140 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee7e75d640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee70005ab0 tx=0x7fee700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee76ffd640 1 -- 172.21.15.28:0/2221991868 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7003f030 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee76ffd640 1 -- 172.21.15.28:0/2221991868 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee70003ce0 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7819f830 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee76ffd640 1 -- 172.21.15.28:0/2221991868 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70035070 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.148+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a7170 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.150+0000 7fee76ffd640 1 -- 172.21.15.28:0/2221991868 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fee70042030 con 0x7fee78109140 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 Processor -- start 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 -- start start 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 0x5604514a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 0x5604514af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514afbb0 0x5604514b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56045140b090 con 0x5604514a52f0 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5604513b2a70 con 0x5604514a49d0 2024-09-19T09:32:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee82599740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560451407050 con 0x5604514afbb0 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.151+0000 7fee76ffd640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 0x7fee600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7df5c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 0x5604514af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7df5c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 0x5604514af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53776/0 (socket says 172.21.15.28:53776) 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 0x5604514a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7df5c640 1 -- 172.21.15.28:0/200411956 learned_addr learned my addr 172.21.15.28:0/200411956 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7e75d640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 0x7fee600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee76ffd640 1 -- 172.21.15.28:0/2221991868 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee700efc80 con 0x7fee78109140 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514afbb0 0x5604514b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 -- 172.21.15.28:0/200411956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514afbb0 msgr2=0x5604514b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514afbb0 0x5604514b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 -- 172.21.15.28:0/200411956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 msgr2=0x5604514af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 0x5604514af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 -- 172.21.15.28:0/200411956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604513bf130 con 0x5604514a49d0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7d75b640 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 0x5604514a4db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00b620 tx=0x7fee6c00b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793bab8714076e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee7e75d640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 0x7fee600a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee700b8d50 tx=0x7fee7008b0c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee5ffff640 1 -- 172.21.15.28:0/200411956 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004230 con 0x5604514a49d0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee5ffff640 1 -- 172.21.15.28:0/200411956 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c0043d0 con 0x5604514a49d0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.152+0000 7fee5ffff640 1 -- 172.21.15.28:0/200411956 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0035f0 con 0x5604514a49d0 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 -- 172.21.15.28:0/200411956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 msgr2=0x5604514a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 0x5604514a4db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00b620 tx=0x7fee6c00b5b0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 -- 172.21.15.28:0/200411956 shutdown_connections 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514afbb0 0x5604514b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a52f0 0x5604514af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 --2- 172.21.15.28:0/200411956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a49d0 0x5604514a4db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 -- 172.21.15.28:0/200411956 >> 172.21.15.28:0/200411956 conn(0x5604514a0f60 msgr2=0x5604514a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 -- 172.21.15.28:0/200411956 shutdown_connections 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.153+0000 7fee82599740 1 -- 172.21.15.28:0/200411956 wait complete. 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 -- start start 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 Processor -- start 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 0x5604514af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 0x5604514ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 0x5604514acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5604514b2780 con 0x5604514afbb0 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5604513c5f20 con 0x5604514a52f0 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604514b2900 con 0x5604514a49d0 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee82599740 10 client.0 _ll_register_callbacks cb 0x56045138f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 0x5604514acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 0x5604514acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53782/0 (socket says 172.21.15.28:53782) 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee7df5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 0x5604514ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee7d75b640 1 -- 172.21.15.28:0/3880696690 learned_addr learned my addr 172.21.15.28:0/3880696690 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.154+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 0x5604514af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 -- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 msgr2=0x5604514af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 0x5604514af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 -- 172.21.15.28:0/3880696690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 msgr2=0x5604514acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 0x5604514acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 -- 172.21.15.28:0/3880696690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045117c2b0 con 0x5604514a52f0 2024-09-19T09:32:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 0x5604514af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee7df5c640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 0x5604514ac760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee68010ad0 tx=0x7fee68010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef870fd6da2960fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6800fe30 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c0043d0 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee680193c0 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee680195c0 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee82599740 10 client.24884 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.155+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604514be0d0 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c0043d0 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fee68025980 con 0x5604514a52f0 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 10 client.24884 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 10 client.24884 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 20 client.24884 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5604514c0770 0x5604514c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 4 client.24884 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560451418600 con 0x5604514c0770 2024-09-19T09:32:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.156+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5604514c0770 0x5604514c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.157+0000 7fee5d7fa640 10 client.24884 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.157+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5604514c0770 0x5604514c1be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7075a5f235ee898d server_cookie=f032d517f1c0a975 in_seq=0 out_seq=0 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee5d7fa640 1 -- 172.21.15.28:0/3880696690 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560451418600 con 0x5604514c0770 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee5d7fa640 10 client.24884 handle_command_reply: tid=0 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 client.24884 shutdown 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 msgr2=0x5604514ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 0x5604514ac760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee68010ad0 tx=0x7fee68010f90 comp rx=0 tx=0).stop 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 shutdown_connections 2024-09-19T09:32:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5604514c0770 0x5604514c1be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514a52f0 0x5604514ac760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514afbb0 0x5604514acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 --2- 172.21.15.28:0/3880696690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a49d0 0x5604514af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee5d7fa640 0 client.24884 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 >> 172.21.15.28:0/3880696690 conn(0x5604514a0f60 msgr2=0x5604514a6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 shutdown_connections 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 1 -- 172.21.15.28:0/3880696690 wait complete. 2024-09-19T09:32:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.158+0000 7fee82599740 20 client.24884 trim_cache size 0 max 16384 2024-09-19T09:32:20.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 Processor -- start 2024-09-19T09:32:20.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 -- start start 2024-09-19T09:32:20.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x5604514a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5604514bff20 con 0x5604514a49d0 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5604514c05f0 con 0x5604514afbb0 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee82599740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604514c0cc0 con 0x5604514a52f0 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53796/0 (socket says 172.21.15.28:53796) 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 -- 172.21.15.28:0/3696646383 learned_addr learned my addr 172.21.15.28:0/3696646383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 -- 172.21.15.28:0/3696646383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 msgr2=0x5604514ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 -- 172.21.15.28:0/3696646383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 msgr2=0x5604514a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x5604514a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 -- 172.21.15.28:0/3696646383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604514ab770 con 0x5604514a49d0 2024-09-19T09:32:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.177+0000 7fee7d75b640 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004a00 tx=0x7fee6c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5969e07306a9e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee5f7fe640 1 -- 172.21.15.28:0/3696646383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c030070 con 0x5604514a49d0 2024-09-19T09:32:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee5f7fe640 1 -- 172.21.15.28:0/3696646383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c005330 con 0x5604514a49d0 2024-09-19T09:32:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee5f7fe640 1 -- 172.21.15.28:0/3696646383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c025040 con 0x5604514a49d0 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 -- 172.21.15.28:0/3696646383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 msgr2=0x5604514c1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004a00 tx=0x7fee6c00bbb0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 -- 172.21.15.28:0/3696646383 shutdown_connections 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x5604514a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 --2- 172.21.15.28:0/3696646383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c1be0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 -- 172.21.15.28:0/3696646383 >> 172.21.15.28:0/3696646383 conn(0x5604514a0f60 msgr2=0x5604514a7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 -- 172.21.15.28:0/3696646383 shutdown_connections 2024-09-19T09:32:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.178+0000 7fee82599740 1 -- 172.21.15.28:0/3696646383 wait complete. 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 -- start start 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 Processor -- start 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x560451528e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5604514c1a60 con 0x5604514a49d0 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5604514c1390 con 0x5604514afbb0 2024-09-19T09:32:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604514c2130 con 0x5604514a52f0 2024-09-19T09:32:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee82599740 10 client.0 _ll_register_callbacks cb 0x7fee68007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee7d75b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53810/0 (socket says 172.21.15.28:53810) 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.179+0000 7fee7d75b640 1 -- 172.21.15.28:0/2636614800 learned_addr learned my addr 172.21.15.28:0/2636614800 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x560451528e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7df5c640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 -- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 msgr2=0x5604514c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 -- 172.21.15.28:0/2636614800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 msgr2=0x560451528e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x560451528e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 -- 172.21.15.28:0/2636614800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604515293e0 con 0x5604514a49d0 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7df5c640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee7d75b640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004b40 tx=0x7fee6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcd38b7155a68f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c030040 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c005330 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fee6c004e50 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee6c005050 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee82599740 10 client.15180 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56045152a6c0 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.180+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c005330 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fee6c0035f0 con 0x5604514a49d0 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:32:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 20 client.15180 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56045152ed30 0x56045152da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 4 client.15180 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "status"}) -- 0x560451412b10 con 0x56045152ed30 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee7df5c640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56045152ed30 0x56045152da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee5d7fa640 10 client.15180 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.181+0000 7fee7df5c640 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56045152ed30 0x56045152da40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c11a5b664e74ce server_cookie=1fdbc4e02b462cba in_seq=0 out_seq=0 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee5d7fa640 1 -- 172.21.15.28:0/2636614800 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560451412b10 con 0x56045152ed30 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee5d7fa640 10 client.15180 handle_command_reply: tid=0 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 client.15180 shutdown 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 msgr2=0x5604514c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004b40 tx=0x7fee6c004ad0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 shutdown_connections 2024-09-19T09:32:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56045152ed30 0x56045152da40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5604514afbb0 0x560451528e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee5d7fa640 0 client.15180 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604514a52f0 0x5604514c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 --2- 172.21.15.28:0/2636614800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5604514a49d0 0x5604514c3580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 >> 172.21.15.28:0/2636614800 conn(0x5604514a0f60 msgr2=0x56045149f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 shutdown_connections 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 1 -- 172.21.15.28:0/2636614800 wait complete. 2024-09-19T09:32:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.182+0000 7fee82599740 20 client.15180 trim_cache size 0 max 16384 2024-09-19T09:32:20.196 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:20.196 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:20.196 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 2, 2024-09-19T09:32:20.196 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24478, 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 89.026979990000001, 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 120.414261622 2024-09-19T09:32:20.197 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 msgr2=0x7fee600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 0x7fee600a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee700b8d50 tx=0x7fee7008b0c0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 msgr2=0x7fee7819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee70005ab0 tx=0x7fee700042f0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 shutdown_connections 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fee600a3240 0x7fee600a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee78109140 0x7fee7819eec0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee781049c0 0x7fee7819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 --2- 172.21.15.28:0/2221991868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee78104010 0x7fee7819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:20.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 >> 172.21.15.28:0/2221991868 conn(0x7fee780ffa50 msgr2=0x7fee78100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:20.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 shutdown_connections 2024-09-19T09:32:20.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:20.183+0000 7fee801c0640 1 -- 172.21.15.28:0/2221991868 wait complete. 2024-09-19T09:32:20.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:20 smithi028 ceph-mon[30972]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:32:20.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:20 smithi078 ceph-mon[35619]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:32:20.867 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:32:20.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:20 smithi038 ceph-mon[35271]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:32:21.124 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:21.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2578258873 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc106f10 msgr2=0x7f5dbc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2578258873 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc106f10 0x7f5dbc10ea90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009ca0 tx=0x7f5db402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2578258873 shutdown_connections 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2578258873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc102790 0x7f5dbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2578258873 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc106f10 0x7f5dbc10ea90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2578258873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc101de0 0x7f5dbc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2578258873 >> 172.21.15.28:0/2578258873 conn(0x7f5dbc0fd8c0 msgr2=0x7f5dbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2578258873 shutdown_connections 2024-09-19T09:32:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.563+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2578258873 wait complete. 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 Processor -- start 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 -- start start 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc101de0 0x7f5dbc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc106f10 0x7f5dbc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc111f70 con 0x7f5dbc102790 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc111df0 con 0x7f5dbc101de0 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc2b11640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1120f0 con 0x7f5dbc106f10 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53828/0 (socket says 172.21.15.28:53828) 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2461297589 learned_addr learned my addr 172.21.15.28:0/2461297589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc101de0 0x7f5dbc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.564+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc106f10 0x7f5dbc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2461297589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc106f10 msgr2=0x7f5dbc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc106f10 0x7f5dbc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2461297589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc101de0 msgr2=0x7f5dbc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc101de0 0x7f5dbc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2461297589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1a6870 con 0x7f5dbc102790 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c920 tx=0x7f5da800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5db9ffb640 1 -- 172.21.15.28:0/2461297589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8007c70 con 0x7f5dbc102790 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5db9ffb640 1 -- 172.21.15.28:0/2461297589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da8007e10 con 0x7f5dbc102790 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1a6b60 con 0x7f5dbc102790 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5db9ffb640 1 -- 172.21.15.28:0/2461297589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8018890 con 0x7f5dbc102790 2024-09-19T09:32:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.565+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a70f0 con 0x7f5dbc102790 2024-09-19T09:32:21.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.568+0000 7f5db9ffb640 1 -- 172.21.15.28:0/2461297589 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5da8018a70 con 0x7f5dbc102790 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 Processor -- start 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5db9ffb640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 0x7f5dac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 -- start start 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 0x7f5dac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c10db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5db9ffb640 1 -- 172.21.15.28:0/2461297589 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5da80cf8c0 con 0x7f5dbc102790 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c112f0 0x55d851c1b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d851b77090 con 0x55d851c1bbb0 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d851b1ea70 con 0x55d851c112f0 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d851b73050 con 0x55d851c109d0 2024-09-19T09:32:21.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.569+0000 7f5dc10ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc10ae640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53844/0 (socket says 172.21.15.28:53844) 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc10ae640 1 -- 172.21.15.28:0/3134175107 learned_addr learned my addr 172.21.15.28:0/3134175107 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c10db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c112f0 0x55d851c1b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 0x7f5dac0a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db402ade0 tx=0x7f5db4031db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 -- 172.21.15.28:0/3134175107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c112f0 msgr2=0x55d851c1b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c112f0 0x55d851c1b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 -- 172.21.15.28:0/3134175107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 msgr2=0x55d851c1dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 -- 172.21.15.28:0/3134175107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d851b2b130 con 0x55d851c109d0 2024-09-19T09:32:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c10db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ab40 tx=0x7f5db000af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a8eaf6dd136b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5da6ffd640 1 -- 172.21.15.28:0/3134175107 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004020 con 0x55d851c109d0 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5da6ffd640 1 -- 172.21.15.28:0/3134175107 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db00041c0 con 0x55d851c109d0 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.570+0000 7f5da6ffd640 1 -- 172.21.15.28:0/3134175107 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00186a0 con 0x55d851c109d0 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3134175107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 msgr2=0x55d851c10db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c10db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ab40 tx=0x7f5db000af40 comp rx=0 tx=0).stop 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3134175107 shutdown_connections 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c112f0 0x55d851c1b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c1bbb0 0x55d851c1dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3134175107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c10db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3134175107 >> 172.21.15.28:0/3134175107 conn(0x55d851c0cf60 msgr2=0x55d851c0d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3134175107 shutdown_connections 2024-09-19T09:32:21.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3134175107 wait complete. 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.571+0000 7f5dc4eea740 1 -- start start 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 Processor -- start 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c1b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d851c1e780 con 0x55d851c112f0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d851b31f20 con 0x55d851c1bbb0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d851c1e900 con 0x55d851c109d0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc4eea740 10 client.0 _ll_register_callbacks cb 0x55d851afb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc08ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c1b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53850/0 (socket says 172.21.15.28:53850) 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2734974563 learned_addr learned my addr 172.21.15.28:0/2734974563 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c18c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2734974563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 msgr2=0x55d851c1b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c1b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 msgr2=0x55d851c18c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c18c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dbbfff640 1 -- 172.21.15.28:0/2734974563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8518e82b0 con 0x55d851c112f0 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.572+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5da8007590 tx=0x7f5da809b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de637f953312a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80a0260 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5da8018d60 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db00041c0 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5da8080020 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 10 client.15192 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d851c2a030 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db00041c0 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5da809b300 con 0x55d851c112f0 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 20 client.15192 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:21.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c2c6b0 0x55d851c2db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 4 client.15192 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:21.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.573+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d851b84600 con 0x55d851c2c6b0 2024-09-19T09:32:21.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.574+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c2c6b0 0x55d851c2db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.574+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c2c6b0 0x55d851c2db20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b765658be66479d server_cookie=d171b57a3abf8c3d in_seq=0 out_seq=0 2024-09-19T09:32:21.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.574+0000 7f5d9bfff640 10 client.15192 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5d9bfff640 1 -- 172.21.15.28:0/2734974563 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d851b84600 con 0x55d851c2c6b0 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5d9bfff640 10 client.15192 handle_command_reply: tid=0 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 client.15192 shutdown 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 msgr2=0x55d851c18700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5da8007590 tx=0x7f5da809b930 comp rx=0 tx=0).stop 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 shutdown_connections 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c2c6b0 0x55d851c2db20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c18c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c18700 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/2734974563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c1b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 >> 172.21.15.28:0/2734974563 conn(0x55d851c0cf60 msgr2=0x55d851c12c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5d9bfff640 0 client.15192 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:21.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 shutdown_connections 2024-09-19T09:32:21.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 1 -- 172.21.15.28:0/2734974563 wait complete. 2024-09-19T09:32:21.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.575+0000 7f5dc4eea740 20 client.15192 trim_cache size 0 max 16384 2024-09-19T09:32:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 Processor -- start 2024-09-19T09:32:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 -- start start 2024-09-19T09:32:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c2d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2be60 con 0x55d851c112f0 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.593+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2c530 con 0x55d851c1bbb0 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2cc00 con 0x55d851c109d0 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dc08ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c2d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53858/0 (socket says 172.21.15.28:53858) 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3396455740 learned_addr learned my addr 172.21.15.28:0/3396455740 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:21.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3396455740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 msgr2=0x55d851c2d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c2d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3396455740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 msgr2=0x55d851c2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c2bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3396455740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d851c11120 con 0x55d851c112f0 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c2d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:21.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.594+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5da8098830 tx=0x7f5da800ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126dba1a79f3afb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.596+0000 7f5da67fc640 1 -- 172.21.15.28:0/3396455740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80a8070 con 0x55d851c112f0 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.596+0000 7f5da67fc640 1 -- 172.21.15.28:0/3396455740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da80a4b20 con 0x55d851c112f0 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.596+0000 7f5da67fc640 1 -- 172.21.15.28:0/3396455740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da809b300 con 0x55d851c112f0 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3396455740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 msgr2=0x55d851c2cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5da8098830 tx=0x7f5da800ff30 comp rx=0 tx=0).stop 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3396455740 shutdown_connections 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c1bbb0 0x55d851c2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c2cd80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3396455740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c109d0 0x55d851c2d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3396455740 >> 172.21.15.28:0/3396455740 conn(0x55d851c0cf60 msgr2=0x55d851c0d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3396455740 shutdown_connections 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.597+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3396455740 wait complete. 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 -- start start 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 Processor -- start 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 0x55d851c30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c1bbb0 0x55d851c31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2d9a0 con 0x55d851c112f0 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2d2d0 con 0x55d851c109d0 2024-09-19T09:32:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d851c2e070 con 0x55d851c1bbb0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc4eea740 10 client.0 _ll_register_callbacks cb 0x7f5da800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53868/0 (socket says 172.21.15.28:53868) 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3027501342 learned_addr learned my addr 172.21.15.28:0/3027501342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc10ae640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c1bbb0 0x55d851c31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.598+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 0x55d851c30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3027501342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c1bbb0 msgr2=0x55d851c31590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c1bbb0 0x55d851c31590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 msgr2=0x55d851c30b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 0x55d851c30b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 -- 172.21.15.28:0/3027501342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d851c94db0 con 0x55d851c112f0 2024-09-19T09:32:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 0x55d851c30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dbbfff640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5da8098310 tx=0x7f5da809a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd134c8c07c39a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80a8050 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5da809b300 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da80a4b20 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5da80a0260 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dc4eea740 10 client.15198 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d851c94db0 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da80a4b20 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5da807ab60 con 0x55d851c112f0 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.599+0000 7f5dc4eea740 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 20 client.15198 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c9a590 0x55d851c992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 4 client.15198 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "status"}) -- 0x55d851b7eb10 con 0x55d851c9a590 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c9a590 0x55d851c992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc08ad640 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c9a590 0x55d851c992a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d2ae1ede004858e server_cookie=2f799176ee94a565 in_seq=0 out_seq=0 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5d9bfff640 10 client.15198 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5d9bfff640 1 -- 172.21.15.28:0/3027501342 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d851b7eb10 con 0x55d851c9a590 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5d9bfff640 10 client.15198 handle_command_reply: tid=0 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 1 client.15198 shutdown 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 msgr2=0x55d851c31050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.600+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5da8098310 tx=0x7f5da809a3c0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 shutdown_connections 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d851c1bbb0 0x55d851c31590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d851c112f0 0x55d851c31050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d851c9a590 0x55d851c992a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5d9bfff640 0 client.15198 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 --2- 172.21.15.28:0/3027501342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d851c109d0 0x55d851c30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 >> 172.21.15.28:0/3027501342 conn(0x55d851c0cf60 msgr2=0x55d851c07ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 shutdown_connections 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 1 -- 172.21.15.28:0/3027501342 wait complete. 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc4eea740 20 client.15198 trim_cache size 0 max 16384 2024-09-19T09:32:21.608 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 1, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24482, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 91.455724841999995, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:32:21.609 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 108.52635138399999 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 msgr2=0x7f5dac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 0x7f5dac0a5780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db402ade0 tx=0x7f5db4031db0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 msgr2=0x7f5dbc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c920 tx=0x7f5da800cde0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 shutdown_connections 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5dac0a32c0 0x7f5dac0a5780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5dbc106f10 0x7f5dbc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5dbc102790 0x7f5dbc074a30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 --2- 172.21.15.28:0/2461297589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dbc101de0 0x7f5dbc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.601+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 >> 172.21.15.28:0/2461297589 conn(0x7f5dbc0fd8c0 msgr2=0x7f5dbc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.602+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 shutdown_connections 2024-09-19T09:32:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:21.602+0000 7f5dc2b11640 1 -- 172.21.15.28:0/2461297589 wait complete. 2024-09-19T09:32:22.481 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:32:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:22.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:22 smithi028 ceph-mon[30972]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:32:22.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:22 smithi078 ceph-mon[35619]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:32:22.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:22 smithi038 ceph-mon[35271]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:32:23.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.188+0000 7f24c0247640 1 -- 172.21.15.28:0/1413496840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8106a80 msgr2=0x7f24b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.188+0000 7f24c0247640 1 --2- 172.21.15.28:0/1413496840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8106a80 0x7f24b8106e60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009930 tx=0x7f24b402a420 comp rx=0 tx=0).stop 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 -- 172.21.15.28:0/1413496840 shutdown_connections 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 --2- 172.21.15.28:0/1413496840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8100a80 0x7f24b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 --2- 172.21.15.28:0/1413496840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8101420 0x7f24b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 --2- 172.21.15.28:0/1413496840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8106a80 0x7f24b8106e60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 -- 172.21.15.28:0/1413496840 >> 172.21.15.28:0/1413496840 conn(0x7f24b80fc7c0 msgr2=0x7f24b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 -- 172.21.15.28:0/1413496840 shutdown_connections 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 -- 172.21.15.28:0/1413496840 wait complete. 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 Processor -- start 2024-09-19T09:32:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.189+0000 7f24c0247640 1 -- start start 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 0x7f24b8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8106a80 0x7f24b819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8110e40 con 0x7f24b8100a80 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8110cc0 con 0x7f24b8101420 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8110fc0 con 0x7f24b8106a80 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53892/0 (socket says 172.21.15.28:53892) 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 -- 172.21.15.28:0/3883209450 learned_addr learned my addr 172.21.15.28:0/3883209450 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 0x7f24b8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24be7e4640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8106a80 0x7f24b819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 -- 172.21.15.28:0/3883209450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8106a80 msgr2=0x7f24b819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8106a80 0x7f24b819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 -- 172.21.15.28:0/3883209450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 msgr2=0x7f24b8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 0x7f24b8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 -- 172.21.15.28:0/3883209450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b819d3f0 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 0x7f24b8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009230 tx=0x7f24b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24aeffd640 1 -- 172.21.15.28:0/3883209450 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b403f030 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24aeffd640 1 -- 172.21.15.28:0/3883209450 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4003ad0 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b819d680 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.190+0000 7f24aeffd640 1 -- 172.21.15.28:0/3883209450 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4035070 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.191+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a4fc0 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.193+0000 7f24aeffd640 1 -- 172.21.15.28:0/3883209450 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f24b4042030 con 0x7f24b8100a80 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 Processor -- start 2024-09-19T09:32:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 -- start start 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda44db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5ed9ab090 con 0x55a5eda4fbb0 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5ed952a70 con 0x55a5eda452f0 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24c2620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5ed9a7050 con 0x55a5eda449d0 2024-09-19T09:32:23.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24bd7e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24bd7e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53898/0 (socket says 172.21.15.28:53898) 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24bd7e2640 1 -- 172.21.15.28:0/1350755267 learned_addr learned my addr 172.21.15.28:0/1350755267 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.194+0000 7f24be7e4640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda44db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 -- 172.21.15.28:0/1350755267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 msgr2=0x55a5eda44db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda44db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 -- 172.21.15.28:0/1350755267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 msgr2=0x55a5eda4f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 -- 172.21.15.28:0/1350755267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ed95f130 con 0x55a5eda4fbb0 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24aeffd640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 0x7f24a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24be7e4640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 0x7f24a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f24a800bcb0 tx=0x7f24a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73b7ada2712bfed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24aeffd640 1 -- 172.21.15.28:0/3883209450 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f24b40efe80 con 0x7f24b8100a80 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24a3fff640 1 -- 172.21.15.28:0/1350755267 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8011440 con 0x55a5eda4fbb0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24a3fff640 1 -- 172.21.15.28:0/1350755267 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8011ac0 con 0x55a5eda4fbb0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.195+0000 7f24a3fff640 1 -- 172.21.15.28:0/1350755267 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a801bb50 con 0x55a5eda4fbb0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.196+0000 7f24c2620740 1 -- 172.21.15.28:0/1350755267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 msgr2=0x55a5eda51fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.196+0000 7f24c2620740 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f24a800bcb0 tx=0x7f24a800f730 comp rx=0 tx=0).stop 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24be7e4640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 0x7f24a40a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005fd0 tx=0x7f24b00096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 -- 172.21.15.28:0/1350755267 shutdown_connections 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda51fa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 --2- 172.21.15.28:0/1350755267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda44db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 -- 172.21.15.28:0/1350755267 >> 172.21.15.28:0/1350755267 conn(0x55a5eda40f60 msgr2=0x55a5eda41340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 -- 172.21.15.28:0/1350755267 shutdown_connections 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 -- 172.21.15.28:0/1350755267 wait complete. 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 -- start start 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.197+0000 7f24c2620740 1 Processor -- start 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5eda4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda52780 con 0x55a5eda449d0 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5ed965f20 con 0x55a5eda452f0 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda52900 con 0x55a5eda4fbb0 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24c2620740 10 client.0 _ll_register_callbacks cb 0x55a5ed92f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:23.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53904/0 (socket says 172.21.15.28:53904) 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 -- 172.21.15.28:0/417774051 learned_addr learned my addr 172.21.15.28:0/417774051 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24be7e4640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5eda4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 -- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 msgr2=0x55a5eda4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5eda4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 -- 172.21.15.28:0/417774051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 msgr2=0x55a5eda4c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.198+0000 7f24bdfe3640 1 -- 172.21.15.28:0/417774051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ed71c2b0 con 0x55a5eda449d0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f24b40bc460 tx=0x7f24b4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfc8765f04905e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:23.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4049070 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f24b40b8b40 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a8011ac0 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24b40bbe30 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 10 client.15210 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5eda5e030 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8011ac0 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f24b4003c80 con 0x55a5eda449d0 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 10 client.15210 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 10 client.15210 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.199+0000 7f24c2620740 20 client.15210 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5eda606d0 0x55a5eda61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24c2620740 4 client.15210 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:23.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ed9b8600 con 0x55a5eda606d0 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5eda606d0 0x55a5eda61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5eda606d0 0x55a5eda61b40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf77db21cdcb2321 server_cookie=2a4d8302e473168f in_seq=0 out_seq=0 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.200+0000 7f24a17fa640 10 client.15210 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24a17fa640 1 -- 172.21.15.28:0/417774051 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5ed9b8600 con 0x55a5eda606d0 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24a17fa640 10 client.15210 handle_command_reply: tid=0 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 client.15210 shutdown 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 msgr2=0x55a5eda4f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f24b40bc460 tx=0x7f24b4009660 comp rx=0 tx=0).stop 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 shutdown_connections 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda4c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5eda606d0 0x55a5eda61b40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5eda4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 --2- 172.21.15.28:0/417774051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda449d0 0x55a5eda4f3e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24a17fa640 0 client.15210 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 >> 172.21.15.28:0/417774051 conn(0x55a5eda40f60 msgr2=0x55a5eda46c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 shutdown_connections 2024-09-19T09:32:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 1 -- 172.21.15.28:0/417774051 wait complete. 2024-09-19T09:32:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.201+0000 7f24c2620740 20 client.15210 trim_cache size 0 max 16384 2024-09-19T09:32:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.219+0000 7f24c2620740 1 Processor -- start 2024-09-19T09:32:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.219+0000 7f24c2620740 1 -- start start 2024-09-19T09:32:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.219+0000 7f24c2620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda61470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24c2620740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24c2620740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24c2620740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda5fe80 con 0x55a5eda4fbb0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24c2620740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda60550 con 0x55a5eda452f0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24c2620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda60c20 con 0x55a5eda449d0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24bdfe3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda61470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53914/0 (socket says 172.21.15.28:53914) 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 -- 172.21.15.28:0/1883230019 learned_addr learned my addr 172.21.15.28:0/1883230019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 -- 172.21.15.28:0/1883230019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 msgr2=0x55a5eda61470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda61470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 -- 172.21.15.28:0/1883230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 msgr2=0x55a5eda60da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda60da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 -- 172.21.15.28:0/1883230019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5eda40d90 con 0x55a5eda4fbb0 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda61470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24be7e4640 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24b001e620 tx=0x7f24b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9251b58e1eefa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.220+0000 7f24a37fe640 1 -- 172.21.15.28:0/1883230019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d4d0 con 0x55a5eda4fbb0 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24a37fe640 1 -- 172.21.15.28:0/1883230019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b001db50 con 0x55a5eda4fbb0 2024-09-19T09:32:23.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24a37fe640 1 -- 172.21.15.28:0/1883230019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0027b80 con 0x55a5eda4fbb0 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 -- 172.21.15.28:0/1883230019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 msgr2=0x55a5eda60000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24b001e620 tx=0x7f24b0015000 comp rx=0 tx=0).stop 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 -- 172.21.15.28:0/1883230019 shutdown_connections 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda452f0 0x55a5eda60da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda4fbb0 0x55a5eda60000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 --2- 172.21.15.28:0/1883230019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda449d0 0x55a5eda61470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 -- 172.21.15.28:0/1883230019 >> 172.21.15.28:0/1883230019 conn(0x55a5eda40f60 msgr2=0x55a5eda3bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 -- 172.21.15.28:0/1883230019 shutdown_connections 2024-09-19T09:32:23.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.221+0000 7f24c2620740 1 -- 172.21.15.28:0/1883230019 wait complete. 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 -- start start 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 Processor -- start 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda449d0 0x55a5eda636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5edac6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda619c0 con 0x55a5eda452f0 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda612f0 con 0x55a5eda449d0 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5eda62090 con 0x55a5eda4fbb0 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24c2620740 10 client.0 _ll_register_callbacks cb 0x7f24b4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24bd7e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24bd7e2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53928/0 (socket says 172.21.15.28:53928) 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24bd7e2640 1 -- 172.21.15.28:0/4213781691 learned_addr learned my addr 172.21.15.28:0/4213781691 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.222+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda449d0 0x55a5eda636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24be7e4640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5edac6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 -- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 msgr2=0x55a5edac6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5edac6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 -- 172.21.15.28:0/4213781691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda449d0 msgr2=0x55a5eda636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda449d0 0x55a5eda636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 -- 172.21.15.28:0/4213781691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5edac7560 con 0x55a5eda452f0 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24bd7e2640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24a801b6c0 tx=0x7f24a801b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7533c8f860d101cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:23.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80054c0 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f24a801b760 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b001db50 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24a8038070 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24c2620740 10 client.15216 fetch_fsmap learned FSMap version 26 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.223+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5edace930 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b001db50 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f24a80039c0 con 0x55a5eda452f0 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 20 client.15216 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5edad3220 0x55a5edad1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 4 client.15216 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:23.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "status"}) -- 0x55a5ed9b2b10 con 0x55a5edad3220 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.224+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5edad3220 0x55a5edad1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24a17fa640 10 client.15216 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24bdfe3640 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5edad3220 0x55a5edad1f30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97664f091e4c5a server_cookie=f78d949101f6f4fa in_seq=0 out_seq=0 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24a17fa640 1 -- 172.21.15.28:0/4213781691 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a5ed9b2b10 con 0x55a5edad3220 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24a17fa640 10 client.15216 handle_command_reply: tid=0 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 client.15216 shutdown 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 msgr2=0x55a5eda63c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24a801b6c0 tx=0x7f24a801b650 comp rx=0 tx=0).stop 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 shutdown_connections 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a5edad3220 0x55a5edad1f30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5eda4fbb0 0x55a5edac6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a5eda452f0 0x55a5eda63c10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24a17fa640 0 client.15216 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 --2- 172.21.15.28:0/4213781691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5eda449d0 0x55a5eda636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 >> 172.21.15.28:0/4213781691 conn(0x55a5eda40f60 msgr2=0x55a5eda5f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 shutdown_connections 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 1 -- 172.21.15.28:0/4213781691 wait complete. 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.225+0000 7f24c2620740 20 client.15216 trim_cache size 0 max 16384 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-19T09:32:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24499, 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 99.672921212000006, 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 118.511752824 2024-09-19T09:32:23.234 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 msgr2=0x7f24a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 0x7f24a40a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005fd0 tx=0x7f24b00096e0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 msgr2=0x7f24b8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009230 tx=0x7f24b4004270 comp rx=0 tx=0).stop 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 shutdown_connections 2024-09-19T09:32:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24a40a3240 0x7f24a40a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24b8101420 0x7f24b8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24b8106a80 0x7f24b819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 --2- 172.21.15.28:0/3883209450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24b8100a80 0x7f24b8198440 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 >> 172.21.15.28:0/3883209450 conn(0x7f24b80fc7c0 msgr2=0x7f24b806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 shutdown_connections 2024-09-19T09:32:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:23.226+0000 7f24c0247640 1 -- 172.21.15.28:0/3883209450 wait complete. 2024-09-19T09:32:23.939 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T09:32:23.949 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T09:32:23.949 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T09:32:23.949 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T09:32:23.949 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T09:32:23.974 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T09:32:23.974 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T09:32:23.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:23.974 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-19T09:32:23.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:23.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-19T09:32:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-09-19T09:32:24.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:32:24.063 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T09:32:24.063 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T09:32:24.063 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:24.064 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T09:32:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:24.749 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:24 smithi028 ceph-mon[30972]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:24.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.771+0000 7fb240fe2640 1 -- 172.21.15.28:0/3549576342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c101280 msgr2=0x7fb23c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:24.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.771+0000 7fb240fe2640 1 --2- 172.21.15.28:0/3549576342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c101280 0x7fb23c10df40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb230009900 tx=0x7fb23002a410 comp rx=0 tx=0).stop 2024-09-19T09:32:24.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 -- 172.21.15.28:0/3549576342 shutdown_connections 2024-09-19T09:32:24.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 --2- 172.21.15.28:0/3549576342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 0x7fb23c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 --2- 172.21.15.28:0/3549576342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c101280 0x7fb23c10df40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 --2- 172.21.15.28:0/3549576342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c1008d0 0x7fb23c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 -- 172.21.15.28:0/3549576342 >> 172.21.15.28:0/3549576342 conn(0x7fb23c0fc7a0 msgr2=0x7fb23c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 -- 172.21.15.28:0/3549576342 shutdown_connections 2024-09-19T09:32:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.772+0000 7fb240fe2640 1 -- 172.21.15.28:0/3549576342 wait complete. 2024-09-19T09:32:24.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 Processor -- start 2024-09-19T09:32:24.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 -- start start 2024-09-19T09:32:24.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c101280 0x7fb23c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 0x7fb23c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1141a0 con 0x7fb23c1008d0 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c114020 con 0x7fb23c101280 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb240fe2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c114320 con 0x7fb23c10e480 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55286/0 (socket says 172.21.15.28:55286) 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 -- 172.21.15.28:0/2761581273 learned_addr learned my addr 172.21.15.28:0/2761581273 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23ad9d640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 0x7fb23c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb239d9b640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c101280 0x7fb23c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 -- 172.21.15.28:0/2761581273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 msgr2=0x7fb23c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 0x7fb23c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 -- 172.21.15.28:0/2761581273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c101280 msgr2=0x7fb23c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.773+0000 7fb23a59c640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c101280 0x7fb23c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb23a59c640 1 -- 172.21.15.28:0/2761581273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c19f6d0 con 0x7fb23c1008d0 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb23a59c640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9f0 tx=0x7fb22c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00cd80 con 0x7fb23c1008d0 2024-09-19T09:32:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb22c010910 con 0x7fb23c1008d0 2024-09-19T09:32:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c19f9c0 con 0x7fb23c1008d0 2024-09-19T09:32:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c018990 con 0x7fb23c1008d0 2024-09-19T09:32:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.774+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a72a0 con 0x7fb23c1008d0 2024-09-19T09:32:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.775+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c06b070 con 0x7fb23c1008d0 2024-09-19T09:32:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.777+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb22c010ac0 con 0x7fb23c1008d0 2024-09-19T09:32:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.778+0000 7fb22b7fe640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 0x7fb2240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.778+0000 7fb239d9b640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 0x7fb2240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:24.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.778+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb22c0d0090 con 0x7fb23c1008d0 2024-09-19T09:32:24.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.778+0000 7fb239d9b640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 0x7fb2240a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb230009230 tx=0x7fb2300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.781+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb22c09c730 con 0x7fb23c1008d0 2024-09-19T09:32:24.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:24 smithi078 ceph-mon[35619]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:24.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:24 smithi038 ceph-mon[35271]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:24.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.996+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb23c100cb0 con 0x7fb23c1008d0 2024-09-19T09:32:24.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.997+0000 7fb22b7fe640 1 -- 172.21.15.28:0/2761581273 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fb23c100cb0 con 0x7fb23c1008d0 2024-09-19T09:32:24.998 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.998+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 msgr2=0x7fb2240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.998+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 0x7fb2240a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb230009230 tx=0x7fb2300058d0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.998+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 msgr2=0x7fb23c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.998+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9f0 tx=0x7fb22c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 shutdown_connections 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb2240a3240 0x7fb2240a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb23c101280 0x7fb23c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c10e480 0x7fb23c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 --2- 172.21.15.28:0/2761581273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb23c1008d0 0x7fb23c19a720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:25.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 >> 172.21.15.28:0/2761581273 conn(0x7fb23c0fc7a0 msgr2=0x7fb23c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:25.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 shutdown_connections 2024-09-19T09:32:25.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:24.999+0000 7fb240fe2640 1 -- 172.21.15.28:0/2761581273 wait complete. 2024-09-19T09:32:25.619 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:25 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2761581273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T09:32:25.649 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:32:25.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:25.650 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-19T09:32:25.667 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42206sec preferred_lft 42206sec 2024-09-19T09:32:25.668 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-19T09:32:25.669 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:32:25.669 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:32:25.669 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-19T09:32:25.669 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-19T09:32:25.669 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T09:32:25.669 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:32:25.669 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-19T09:32:25.669 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2024-09-19T09:32:25.670 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2024-09-19T09:32:25.670 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2024-09-19T09:32:25.670 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T09:32:25.670 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-19T09:32:25.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:25.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:25 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2761581273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:32:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:25.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:25.880 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T09:32:25.906 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-19T09:32:25.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:25.908 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-19T09:32:25.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:25 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2761581273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:32:25.975 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-19T09:32:25.976 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T09:32:25.976 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T09:32:25.976 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T09:32:25.976 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T09:32:25.976 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:32:25.976 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-19T09:32:25.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T09:32:25.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T09:32:25.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T09:32:25.977 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-19T09:32:26.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:26.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:26.276 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-19T09:32:26.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:26.292 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-09-19T09:32:26.349 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:32:26.349 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-19T09:32:26.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:32:26.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T09:32:26.349 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-19T09:32:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:26.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:26.469 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T09:32:26.469 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-19T09:32:26.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:26.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:26.615 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:32:26.616 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-19T09:32:26.616 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-19T09:32:26.616 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T09:32:26.616 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-19T09:32:26.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:26.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:32:26.695 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:26.711 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:32:26.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:32:26.711 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:26.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:26 smithi028 ceph-mon[30972]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:26.767 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T09:32:26.767 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T09:32:26.767 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391 2024-09-19T09:32:26.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:26.768 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:32:26.864 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:32:26.864 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-19T09:32:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:26 smithi038 ceph-mon[35271]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:26.991 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.38:3300,172.21.15.78:3300" 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with new device syntax: 0@9a2e9aa4-7668-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.28:3300/172.21.15.38:3300/172.21.15.78:3300" will pass to kernel 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T09:32:26.992 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-19T09:32:26.993 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T09:32:26.993 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T09:32:26.993 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T09:32:26.994 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T09:32:26.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:32:26.994 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:27.021 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:32:27.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:26 smithi078 ceph-mon[35619]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:32:27.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:32:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:27.119 INFO:teuthology.orchestra.run.smithi028.stdout:client24926 2024-09-19T09:32:27.119 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:32:27.119 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:32:27.137 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:32:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:32:27.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:27.234 INFO:teuthology.orchestra.run.smithi028.stdout:client24926 2024-09-19T09:32:27.234 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:32:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:32:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:27.291 INFO:teuthology.orchestra.run.smithi028.stdout:client24926 2024-09-19T09:32:27.291 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968.client24926/status') 2024-09-19T09:32:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:32:27.352 INFO:teuthology.orchestra.run.smithi028.stdout:instance: client.24926 (3)172.21.15.28:0/181458535 2024-09-19T09:32:27.352 INFO:teuthology.orchestra.run.smithi028.stdout:blocklisted: false 2024-09-19T09:32:27.352 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records in 2024-09-19T09:32:27.352 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records out 2024-09-19T09:32:27.352 INFO:teuthology.orchestra.run.smithi028.stderr:70 bytes copied, 0.000258609 s, 271 kB/s 2024-09-19T09:32:27.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:32:27.360 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T09:32:27.369 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T09:32:27.369 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:32:27.369 DEBUG:teuthology.orchestra.run.smithi028:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.472+0000 7f284843c640 1 -- 172.21.15.28:0/222246993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840100a70 msgr2=0x7f2840100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.472+0000 7f284843c640 1 --2- 172.21.15.28:0/222246993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840100a70 0x7f2840100e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f283c009900 tx=0x7f283c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 -- 172.21.15.28:0/222246993 shutdown_connections 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 --2- 172.21.15.28:0/222246993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840101c60 0x7f28401020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 --2- 172.21.15.28:0/222246993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840102620 0x7f2840108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 --2- 172.21.15.28:0/222246993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840100a70 0x7f2840100e70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 -- 172.21.15.28:0/222246993 >> 172.21.15.28:0/222246993 conn(0x7f28400fc1f0 msgr2=0x7f28400fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 -- 172.21.15.28:0/222246993 shutdown_connections 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 -- 172.21.15.28:0/222246993 wait complete. 2024-09-19T09:32:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 Processor -- start 2024-09-19T09:32:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 -- start start 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.473+0000 7f284843c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840100a70 0x7f2840197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840101c60 0x7f28401983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f284010c720 con 0x7f2840102620 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f284010c5a0 con 0x7f2840100a70 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f284010c8a0 con 0x7f2840101c60 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55310/0 (socket says 172.21.15.28:55310) 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 -- 172.21.15.28:0/2141120596 learned_addr learned my addr 172.21.15.28:0/2141120596 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28461d8640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840100a70 0x7f2840197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28459d7640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840101c60 0x7f28401983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 -- 172.21.15.28:0/2141120596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840101c60 msgr2=0x7f28401983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840101c60 0x7f28401983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 -- 172.21.15.28:0/2141120596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840100a70 msgr2=0x7f2840197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840100a70 0x7f2840197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 -- 172.21.15.28:0/2141120596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284019fac0 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28469d9640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f283800cce0 tx=0x7f283800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283800f450 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284019fdb0 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283800fad0 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.474+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838018c10 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.475+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a0220 con 0x7f2840102620 2024-09-19T09:32:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.476+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28401a04e0 con 0x7f2840102620 2024-09-19T09:32:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.477+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f283800f5f0 con 0x7f2840102620 2024-09-19T09:32:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.477+0000 7f28377fe640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 0x7f282c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.477+0000 7f28461d8640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 0x7f282c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.477+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f28380cffd0 con 0x7f2840102620 2024-09-19T09:32:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.478+0000 7f28461d8640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 0x7f282c0a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f283c005ab0 tx=0x7f283c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.482+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2838014030 con 0x7f2840102620 2024-09-19T09:32:27.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:27.696+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f28401a06b0 con 0x7f2840102620 2024-09-19T09:32:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.621+0000 7f28377fe640 1 -- 172.21.15.28:0/2141120596 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f283809cd20 con 0x7f2840102620 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 msgr2=0x7f282c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 0x7f282c0a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f283c005ab0 tx=0x7f283c005a20 comp rx=0 tx=0).stop 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 msgr2=0x7f284019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f283800cce0 tx=0x7f283800a620 comp rx=0 tx=0).stop 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 shutdown_connections 2024-09-19T09:32:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f282c0a3240 0x7f282c0a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2840102620 0x7f284019f450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2840101c60 0x7f28401983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 --2- 172.21.15.28:0/2141120596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2840100a70 0x7f2840197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.624+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 >> 172.21.15.28:0/2141120596 conn(0x7f28400fc1f0 msgr2=0x7f28400fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.625+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 shutdown_connections 2024-09-19T09:32:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.625+0000 7f284843c640 1 -- 172.21.15.28:0/2141120596 wait complete. 2024-09-19T09:32:28.640 DEBUG:teuthology.orchestra.run.smithi028:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- 172.21.15.28:0/3035694457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 msgr2=0x7f53a0108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- 172.21.15.28:0/3035694457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a0108eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5398009be0 tx=0x7f539802a6c0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- 172.21.15.28:0/3035694457 shutdown_connections 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- 172.21.15.28:0/3035694457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0101c60 0x7f53a01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- 172.21.15.28:0/3035694457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a0108eb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- 172.21.15.28:0/3035694457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0100a70 0x7f53a0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- 172.21.15.28:0/3035694457 >> 172.21.15.28:0/3035694457 conn(0x7f53a00fc1f0 msgr2=0x7f53a00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- 172.21.15.28:0/3035694457 shutdown_connections 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- 172.21.15.28:0/3035694457 wait complete. 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 Processor -- start 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 -- start start 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0100a70 0x7f53a006ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 0x7f53a019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.752+0000 7f53a60d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f53a60d7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c6a0 con 0x7f53a0102620 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f53a60d7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c520 con 0x7f53a0100a70 2024-09-19T09:32:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f53a60d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c820 con 0x7f53a0101c60 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55318/0 (socket says 172.21.15.28:55318) 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 -- 172.21.15.28:0/2051926859 learned_addr learned my addr 172.21.15.28:0/2051926859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539f7fe640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0100a70 0x7f53a006ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f5397fff640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 0x7f53a019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 -- 172.21.15.28:0/2051926859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 msgr2=0x7f53a019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 0x7f53a019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 -- 172.21.15.28:0/2051926859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0100a70 msgr2=0x7f53a006ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0100a70 0x7f53a006ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 -- 172.21.15.28:0/2051926859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a019fb00 con 0x7f53a0102620 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f5397fff640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 0x7f53a019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539ffff640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5398009d00 tx=0x7f5398030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398041020 con 0x7f53a0102620 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5398037040 con 0x7f53a0102620 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a019fd30 con 0x7f53a0102620 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398036070 con 0x7f53a0102620 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.753+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a01a01d0 con 0x7f53a0102620 2024-09-19T09:32:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.754+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a0107e60 con 0x7f53a0102620 2024-09-19T09:32:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.756+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5398044020 con 0x7f53a0102620 2024-09-19T09:32:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.756+0000 7f539d7fa640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 0x7f538c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.757+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f53980f0910 con 0x7f53a0102620 2024-09-19T09:32:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.757+0000 7f539f7fe640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 0x7f538c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.757+0000 7f539f7fe640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 0x7f538c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53a0101ac0 tx=0x7f5390005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.760+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f539804b020 con 0x7f53a0102620 2024-09-19T09:32:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:28 smithi038 ceph-mon[35271]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:32:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:28 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:32:28.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.963+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f53a0062790 con 0x7f538c0a3240 2024-09-19T09:32:28.971 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-mon[30972]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:32:28.971 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:32:28.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.983+0000 7f539d7fa640 1 -- 172.21.15.28:0/2051926859 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f53a0062790 con 0x7f538c0a3240 2024-09-19T09:32:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:{} 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 msgr2=0x7f538c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 0x7f538c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53a0101ac0 tx=0x7f5390005f90 comp rx=0 tx=0).stop 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 msgr2=0x7f53a019f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5398009d00 tx=0x7f5398030620 comp rx=0 tx=0).stop 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 shutdown_connections 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f538c0a3240 0x7f538c0a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53a0102620 0x7f53a019f1f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0101c60 0x7f53a019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 --2- 172.21.15.28:0/2051926859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a0100a70 0x7f53a006ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 >> 172.21.15.28:0/2051926859 conn(0x7f53a00fc1f0 msgr2=0x7f53a0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 shutdown_connections 2024-09-19T09:32:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:28.986+0000 7f53a60d7640 1 -- 172.21.15.28:0/2051926859 wait complete. 2024-09-19T09:32:28.999 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T09:32:29.009 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T09:32:29.026 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:32:29.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:28 smithi078 ceph-mon[35619]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:32:29.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:28 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- 172.21.15.28:0/2770440226 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 msgr2=0x7fc820108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 --2- 172.21.15.28:0/2770440226 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc820108e90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc818009ca0 tx=0x7fc81802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- 172.21.15.28:0/2770440226 shutdown_connections 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 --2- 172.21.15.28:0/2770440226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 0x7fc8201020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 --2- 172.21.15.28:0/2770440226 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc820108e90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 --2- 172.21.15.28:0/2770440226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 0x7fc820100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- 172.21.15.28:0/2770440226 >> 172.21.15.28:0/2770440226 conn(0x7fc8200fc190 msgr2=0x7fc8200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- 172.21.15.28:0/2770440226 shutdown_connections 2024-09-19T09:32:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- 172.21.15.28:0/2770440226 wait complete. 2024-09-19T09:32:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 Processor -- start 2024-09-19T09:32:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.129+0000 7fc82664c640 1 -- start start 2024-09-19T09:32:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 0x7fc82019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 0x7fc82019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010b650 con 0x7fc820102600 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010b4d0 con 0x7fc820101c40 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc82664c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010b7d0 con 0x7fc820100a50 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55368/0 (socket says 172.21.15.28:55368) 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 -- 172.21.15.28:0/1634505943 learned_addr learned my addr 172.21.15.28:0/1634505943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc81ffff640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 0x7fc82019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc81f7fe640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 0x7fc82019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 -- 172.21.15.28:0/1634505943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 msgr2=0x7fc82019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 0x7fc82019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 -- 172.21.15.28:0/1634505943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 msgr2=0x7fc82019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 0x7fc82019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 -- 172.21.15.28:0/1634505943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8201a1cc0 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.130+0000 7fc824be9640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8180319f0 tx=0x7fc81802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.131+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818003bf0 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.131+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8201a1f50 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.131+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818003d90 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.131+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81802fd00 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.131+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8201a2410 con 0x7fc820102600 2024-09-19T09:32:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.132+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc820107d10 con 0x7fc820102600 2024-09-19T09:32:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.133+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc818004040 con 0x7fc820102600 2024-09-19T09:32:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.134+0000 7fc81d7fa640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 0x7fc8080a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.134+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc8180f1100 con 0x7fc820102600 2024-09-19T09:32:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.134+0000 7fc81ffff640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 0x7fc8080a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.134+0000 7fc81ffff640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 0x7fc8080a5710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8100058c0 tx=0x7fc81000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.137+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8180bddf0 con 0x7fc820102600 2024-09-19T09:32:29.264 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:32:28.968+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:32:29.264 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:32:28.968+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:32:29.264 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:32:28.968+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:32:29.264 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:32:28.968+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:32:29.264 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:32:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x[31199]: 2024-09-19T09:32:28.968+0000 7fbac6e5f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:32:29.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.315+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc82002da10 con 0x7fc820102600 2024-09-19T09:32:29.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.316+0000 7fc81d7fa640 1 -- 172.21.15.28:0/1634505943 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22580 (secure 0 0 0) 0x7fc8180bd6f0 con 0x7fc820102600 2024-09-19T09:32:29.320 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:29.320 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:31:49.348824+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:32:29.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 msgr2=0x7fc8080a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 0x7fc8080a5710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8100058c0 tx=0x7fc81000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 msgr2=0x7fc8201a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc8180319f0 tx=0x7fc81802f380 comp rx=0 tx=0).stop 2024-09-19T09:32:29.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 shutdown_connections 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc8080a3250 0x7fc8080a5710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc820101c40 0x7fc82019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc820102600 0x7fc8201a1650 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 --2- 172.21.15.28:0/1634505943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100a50 0x7fc82019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 >> 172.21.15.28:0/1634505943 conn(0x7fc8200fc190 msgr2=0x7fc820104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 shutdown_connections 2024-09-19T09:32:29.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.319+0000 7fc82664c640 1 -- 172.21.15.28:0/1634505943 wait complete. 2024-09-19T09:32:29.332 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:26:05.815869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:30:39.464809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:30:41.429176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:32:29.332 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:32:29.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 -- 172.21.15.28:0/3935448218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78104f00 msgr2=0x7fae781072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 --2- 172.21.15.28:0/3935448218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78104f00 0x7fae781072f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009900 tx=0x7fae6c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 -- 172.21.15.28:0/3935448218 shutdown_connections 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 --2- 172.21.15.28:0/3935448218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78107830 0x7fae78109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 --2- 172.21.15.28:0/3935448218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78104f00 0x7fae781072f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 --2- 172.21.15.28:0/3935448218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae781049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 -- 172.21.15.28:0/3935448218 >> 172.21.15.28:0/3935448218 conn(0x7fae780fc190 msgr2=0x7fae780fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 -- 172.21.15.28:0/3935448218 shutdown_connections 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.437+0000 7fae7d68b640 1 -- 172.21.15.28:0/3935448218 wait complete. 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 Processor -- start 2024-09-19T09:32:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 -- start start 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae78101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78104f00 0x7fae78100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d800 con 0x7fae78107830 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d680 con 0x7fae78104f00 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae7d68b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae7810d980 con 0x7fae781025d0 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55390/0 (socket says 172.21.15.28:55390) 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 -- 172.21.15.28:0/1480254985 learned_addr learned my addr 172.21.15.28:0/1480254985 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae76ffd640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae78101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae767fc640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78104f00 0x7fae78100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 -- 172.21.15.28:0/1480254985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 msgr2=0x7fae78101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae78101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 -- 172.21.15.28:0/1480254985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78104f00 msgr2=0x7fae78100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78104f00 0x7fae78100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae777fe640 1 -- 172.21.15.28:0/1480254985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae78101350 con 0x7fae78107830 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.438+0000 7fae76ffd640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae78101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae777fe640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae7000cab0 tx=0x7fae7000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7000ae20 con 0x7fae78107830 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae781a2090 con 0x7fae78107830 2024-09-19T09:32:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7000f930 con 0x7fae78107830 2024-09-19T09:32:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70018b80 con 0x7fae78107830 2024-09-19T09:32:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.439+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae781a2560 con 0x7fae78107830 2024-09-19T09:32:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.440+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae781052e0 con 0x7fae78107830 2024-09-19T09:32:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.441+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fae7000fae0 con 0x7fae78107830 2024-09-19T09:32:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.442+0000 7fae5ffff640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 0x7fae600a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.442+0000 7fae76ffd640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 0x7fae600a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.442+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fae700d01a0 con 0x7fae78107830 2024-09-19T09:32:29.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.442+0000 7fae76ffd640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 0x7fae600a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae68005fd0 tx=0x7fae68005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.446+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae70014030 con 0x7fae78107830 2024-09-19T09:32:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.626+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae78062df0 con 0x7fae78107830 2024-09-19T09:32:29.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.626+0000 7fae5ffff640 1 -- 172.21.15.28:0/1480254985 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fae7009ce90 con 0x7fae78107830 2024-09-19T09:32:29.628 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-19T09:32:29.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 msgr2=0x7fae600a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 0x7fae600a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae68005fd0 tx=0x7fae68005ea0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 msgr2=0x7fae78100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae7000cab0 tx=0x7fae7000cf70 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 shutdown_connections 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fae600a3250 0x7fae600a5710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae78104f00 0x7fae78100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae78107830 0x7fae78100ac0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 --2- 172.21.15.28:0/1480254985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae781025d0 0x7fae78101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.628+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 >> 172.21.15.28:0/1480254985 conn(0x7fae780fc190 msgr2=0x7fae78103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.629+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 shutdown_connections 2024-09-19T09:32:29.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.629+0000 7fae7d68b640 1 -- 172.21.15.28:0/1480254985 wait complete. 2024-09-19T09:32:29.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:32:29.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- 172.21.15.28:0/1204039402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 msgr2=0x7f0f7c1049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- 172.21.15.28:0/1204039402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c1049c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009900 tx=0x7f0f6c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- 172.21.15.28:0/1204039402 shutdown_connections 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- 172.21.15.28:0/1204039402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c104f00 0x7f0f7c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- 172.21.15.28:0/1204039402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c107830 0x7f0f7c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- 172.21.15.28:0/1204039402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c1049c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- 172.21.15.28:0/1204039402 >> 172.21.15.28:0/1204039402 conn(0x7f0f7c0fc190 msgr2=0x7f0f7c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- 172.21.15.28:0/1204039402 shutdown_connections 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- 172.21.15.28:0/1204039402 wait complete. 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 Processor -- start 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- start start 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c104f00 0x7f0f7c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c107830 0x7f0f7c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c10d840 con 0x7f0f7c1025d0 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c10d6c0 con 0x7f0f7c107830 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.752+0000 7f0f82f86640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c10d9c0 con 0x7f0f7c104f00 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55396/0 (socket says 172.21.15.28:55396) 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 -- 172.21.15.28:0/4283130112 learned_addr learned my addr 172.21.15.28:0/4283130112 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f81523640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c107830 0x7f0f7c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f7bfff640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c104f00 0x7f0f7c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 -- 172.21.15.28:0/4283130112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c104f00 msgr2=0x7f0f7c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c104f00 0x7f0f7c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 -- 172.21.15.28:0/4283130112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c107830 msgr2=0x7f0f7c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c107830 0x7f0f7c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 -- 172.21.15.28:0/4283130112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c19fb70 con 0x7f0f7c1025d0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f80d22640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005940 tx=0x7f0f6c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c03f030 con 0x7f0f7c1025d0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c003ad0 con 0x7f0f7c1025d0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c19fe00 con 0x7f0f7c1025d0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.753+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c035070 con 0x7f0f7c1025d0 2024-09-19T09:32:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.754+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c1a0200 con 0x7f0f7c1025d0 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.754+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c0fdc80 con 0x7f0f7c1025d0 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.756+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0f6c042030 con 0x7f0f7c1025d0 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.756+0000 7f0f79ffb640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 0x7f0f640a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.757+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0f6c0f01e0 con 0x7f0f7c1025d0 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.757+0000 7f0f7bfff640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 0x7f0f640a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:29.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.757+0000 7f0f7bfff640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 0x7f0f640a57a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c199460 tx=0x7f0f7000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:29.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.760+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0f6c0bc720 con 0x7f0f7c1025d0 2024-09-19T09:32:29.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:29 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:32:29.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:29 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:32:29.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:29 smithi038 ceph-mon[35271]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:29.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:29 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1634505943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.935+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f7c062e90 con 0x7f0f7c1025d0 2024-09-19T09:32:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.936+0000 7f0f79ffb640 1 -- 172.21.15.28:0/4283130112 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f0f6c0c0a20 con 0x7f0f7c1025d0 2024-09-19T09:32:29.938 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-09-19T09:32:29.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 msgr2=0x7f0f640a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 0x7f0f640a57a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c199460 tx=0x7f0f7000b390 comp rx=0 tx=0).stop 2024-09-19T09:32:29.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 msgr2=0x7f0f7c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:29.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005940 tx=0x7f0f6c031b60 comp rx=0 tx=0).stop 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 shutdown_connections 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0f640a32e0 0x7f0f640a57a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c107830 0x7f0f7c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f7c104f00 0x7f0f7c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 --2- 172.21.15.28:0/4283130112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f7c1025d0 0x7f0f7c197f50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 >> 172.21.15.28:0/4283130112 conn(0x7f0f7c0fc190 msgr2=0x7f0f7c103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 shutdown_connections 2024-09-19T09:32:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:29.938+0000 7f0f82f86640 1 -- 172.21.15.28:0/4283130112 wait complete. 2024-09-19T09:32:29.950 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:29 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:32:29.950 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:29 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:32:29.950 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:29 smithi028 ceph-mon[30972]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:29.950 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:29 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1634505943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:29.951 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:32:30.008 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:29 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2141120596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:32:30.008 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:29 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:32:30.008 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:29 smithi078 ceph-mon[35619]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:32:30.008 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:29 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1634505943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 -- 172.21.15.28:0/2027711942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a4100d00 msgr2=0x7f58a4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/2027711942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a4100d00 0x7f58a4101160 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5894009900 tx=0x7f589402a410 comp rx=0 tx=0).stop 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 -- 172.21.15.28:0/2027711942 shutdown_connections 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/2027711942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/2027711942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a4100d00 0x7f58a4101160 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/2027711942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a41003c0 0x7f58a41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 -- 172.21.15.28:0/2027711942 >> 172.21.15.28:0/2027711942 conn(0x7f58a40fc190 msgr2=0x7f58a40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 -- 172.21.15.28:0/2027711942 shutdown_connections 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.055+0000 7f58ac0f1640 1 -- 172.21.15.28:0/2027711942 wait complete. 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 Processor -- start 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 -- start start 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a4100d00 0x7f58a419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a41a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410f9a0 con 0x7f58a41003c0 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410f820 con 0x7f58a4100d00 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58ac0f1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410fb20 con 0x7f58a41016a0 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58a9e8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58a9e8d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55420/0 (socket says 172.21.15.28:55420) 2024-09-19T09:32:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58a9e8d640 1 -- 172.21.15.28:0/1546176632 learned_addr learned my addr 172.21.15.28:0/1546176632 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58a968c640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a4100d00 0x7f58a419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.056+0000 7f58aa68e640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a41a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 -- 172.21.15.28:0/1546176632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 msgr2=0x7f58a41a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a41a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 -- 172.21.15.28:0/1546176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a4100d00 msgr2=0x7f58a419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a4100d00 0x7f58a419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 -- 172.21.15.28:0/1546176632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a1c60 con 0x7f58a41003c0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58aa68e640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a41a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58a9e8d640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f58a000d950 tx=0x7f58a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0017030 con 0x7f58a41003c0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a0004020 con 0x7f58a41003c0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00047a0 con 0x7f58a41003c0 2024-09-19T09:32:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a1f50 con 0x7f58a41003c0 2024-09-19T09:32:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.057+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a23c0 con 0x7f58a41003c0 2024-09-19T09:32:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.058+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a410b110 con 0x7f58a41003c0 2024-09-19T09:32:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.059+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f58a00041d0 con 0x7f58a41003c0 2024-09-19T09:32:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.060+0000 7f589affd640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 0x7f58900a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.060+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f58a00cfb20 con 0x7f58a41003c0 2024-09-19T09:32:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.060+0000 7f58a968c640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 0x7f58900a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.061+0000 7f58a968c640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 0x7f58900a5820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f58940315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:30.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.063+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58a009c730 con 0x7f58a41003c0 2024-09-19T09:32:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.240+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f58a402da50 con 0x7f58a41003c0 2024-09-19T09:32:30.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.240+0000 7f589affd640 1 -- 172.21.15.28:0/1546176632 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f58a009c030 con 0x7f58a41003c0 2024-09-19T09:32:30.242 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-09-19T09:32:30.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 msgr2=0x7f58900a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 0x7f58900a5820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f58940315b0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 msgr2=0x7f58a419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f58a000d950 tx=0x7f58a000de10 comp rx=0 tx=0).stop 2024-09-19T09:32:30.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 shutdown_connections 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f58900a3360 0x7f58900a5820 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a4100d00 0x7f58a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a41016a0 0x7f58a41a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 --2- 172.21.15.28:0/1546176632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a41003c0 0x7f58a419a040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 >> 172.21.15.28:0/1546176632 conn(0x7f58a40fc190 msgr2=0x7f58a40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 shutdown_connections 2024-09-19T09:32:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.242+0000 7f58ac0f1640 1 -- 172.21.15.28:0/1546176632 wait complete. 2024-09-19T09:32:30.255 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T09:32:30.255 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:30.510 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:30.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:30 smithi038 ceph-mon[35271]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:32:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1480254985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:30 smithi038 ceph-mon[35271]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:32:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4283130112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:30 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1546176632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:30 smithi028 ceph-mon[30972]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:32:30.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:30 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1480254985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:30 smithi028 ceph-mon[30972]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:32:30.934 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:30 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4283130112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.934 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:30 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1546176632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 -- 172.21.15.28:0/3129183921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f272010c170 msgr2=0x7f272010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 --2- 172.21.15.28:0/3129183921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f272010c170 0x7f272010e610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2718009ca0 tx=0x7f271802a950 comp rx=0 tx=0).stop 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 -- 172.21.15.28:0/3129183921 shutdown_connections 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 --2- 172.21.15.28:0/3129183921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f272010c170 0x7f272010e610 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 --2- 172.21.15.28:0/3129183921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720103660 0x7f272010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 --2- 172.21.15.28:0/3129183921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2720102d40 0x7f2720103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 -- 172.21.15.28:0/3129183921 >> 172.21.15.28:0/3129183921 conn(0x7f27200fc7a0 msgr2=0x7f27200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:30.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 -- 172.21.15.28:0/3129183921 shutdown_connections 2024-09-19T09:32:30.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.948+0000 7f2727fd5640 1 -- 172.21.15.28:0/3129183921 wait complete. 2024-09-19T09:32:30.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 Processor -- start 2024-09-19T09:32:30.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 -- start start 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720102d40 0x7f272019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272010c170 0x7f272019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111f70 con 0x7f2720103660 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111df0 con 0x7f272010c170 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2727fd5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27201120f0 con 0x7f2720102d40 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2725570640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2725570640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55438/0 (socket says 172.21.15.28:55438) 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2725570640 1 -- 172.21.15.28:0/4037731196 learned_addr learned my addr 172.21.15.28:0/4037731196 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2725d71640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720102d40 0x7f272019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.949+0000 7f2726572640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272010c170 0x7f272019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 -- 172.21.15.28:0/4037731196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720102d40 msgr2=0x7f272019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720102d40 0x7f272019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 -- 172.21.15.28:0/4037731196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272010c170 msgr2=0x7f272019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272010c170 0x7f272019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 -- 172.21.15.28:0/4037731196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272019f590 con 0x7f2720103660 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2725570640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f271000c970 tx=0x7f271000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710007bb0 con 0x7f2720103660 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2710007d50 con 0x7f2720103660 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271000f450 con 0x7f2720103660 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272019f880 con 0x7f2720103660 2024-09-19T09:32:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.950+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a7160 con 0x7f2720103660 2024-09-19T09:32:30.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.951+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2720103120 con 0x7f2720103660 2024-09-19T09:32:30.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.952+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f2710004f60 con 0x7f2720103660 2024-09-19T09:32:30.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.953+0000 7f2716ffd640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 0x7f270c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:30.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.953+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f27100cf5b0 con 0x7f2720103660 2024-09-19T09:32:30.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.953+0000 7f2725d71640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 0x7f270c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:30.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.954+0000 7f2725d71640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 0x7f270c0a5810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bd0 tx=0x7f271c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:30.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:30.957+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2710014030 con 0x7f2720103660 2024-09-19T09:32:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:30 smithi078 ceph-mon[35619]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:32:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:30 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1480254985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:32:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:30 smithi078 ceph-mon[35619]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:32:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:30 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4283130112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:32:31.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:30 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1546176632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:32:31.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.167+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27201a7490 con 0x7f2720103660 2024-09-19T09:32:31.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.168+0000 7f2716ffd640 1 -- 172.21.15.28:0/4037731196 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f271009bb50 con 0x7f2720103660 2024-09-19T09:32:31.170 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:31.170 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:32:31.171 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:32:31.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 msgr2=0x7f270c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:31.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 0x7f270c0a5810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f271c009bd0 tx=0x7f271c005f90 comp rx=0 tx=0).stop 2024-09-19T09:32:31.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 msgr2=0x7f272019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:31.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f271000c970 tx=0x7f271000ce30 comp rx=0 tx=0).stop 2024-09-19T09:32:31.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 shutdown_connections 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f270c0a3350 0x7f270c0a5810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272010c170 0x7f272019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2720103660 0x7f272019ab20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 --2- 172.21.15.28:0/4037731196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2720102d40 0x7f272019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 >> 172.21.15.28:0/4037731196 conn(0x7f27200fc7a0 msgr2=0x7f27200fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 shutdown_connections 2024-09-19T09:32:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:31.170+0000 7f2727fd5640 1 -- 172.21.15.28:0/4037731196 wait complete. 2024-09-19T09:32:31.900 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:31 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4037731196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:31.901 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T09:32:31.902 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T09:32:31.902 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:32:31.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:31 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4037731196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:32.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/2709222761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 msgr2=0x7f0a88108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/2709222761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a88108e90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009ca0 tx=0x7f0a8002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/2709222761 shutdown_connections 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/2709222761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88101c40 0x7f0a881020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/2709222761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a88108e90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/2709222761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88100a50 0x7f0a88100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.009+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/2709222761 >> 172.21.15.28:0/2709222761 conn(0x7f0a880fc190 msgr2=0x7f0a880fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/2709222761 shutdown_connections 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/2709222761 wait complete. 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 Processor -- start 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- start start 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88100a50 0x7f0a8806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88101c40 0x7f0a8819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8810c6d0 con 0x7f0a88102600 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8810c550 con 0x7f0a88100a50 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.010+0000 7f0a8fe7b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8810c850 con 0x7f0a88101c40 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55448/0 (socket says 172.21.15.28:55448) 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 -- 172.21.15.28:0/3629012360 learned_addr learned my addr 172.21.15.28:0/3629012360 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8dc17640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88100a50 0x7f0a8806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8d416640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88101c40 0x7f0a8819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 -- 172.21.15.28:0/3629012360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88101c40 msgr2=0x7f0a8819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88101c40 0x7f0a8819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 -- 172.21.15.28:0/3629012360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88100a50 msgr2=0x7f0a8806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88100a50 0x7f0a8806aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 -- 172.21.15.28:0/3629012360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8819fa50 con 0x7f0a88102600 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8e418640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0a80031ae0 tx=0x7f0a80031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80003c30 con 0x7f0a88102600 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a80003dd0 con 0x7f0a88102600 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8819fc80 con 0x7f0a88102600 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002fcb0 con 0x7f0a88102600 2024-09-19T09:32:32.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.011+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a881a01d0 con 0x7f0a88102600 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.012+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a88107d70 con 0x7f0a88102600 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.014+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0a80004050 con 0x7f0a88102600 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.014+0000 7f0a7effd640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 0x7f0a740a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.014+0000 7f0a8dc17640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 0x7f0a740a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.014+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0a800f0db0 con 0x7f0a88102600 2024-09-19T09:32:32.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.015+0000 7f0a8dc17640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 0x7f0a740a57a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a88101aa0 tx=0x7f0a84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.017+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a800bdab0 con 0x7f0a88102600 2024-09-19T09:32:32.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:31 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4037731196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:32.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.203+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a88062630 con 0x7f0a740a32e0 2024-09-19T09:32:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.210+0000 7f0a7effd640 1 -- 172.21.15.28:0/3629012360 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7f0a88062630 con 0x7f0a740a32e0 2024-09-19T09:32:32.213 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:32.214 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 msgr2=0x7f0a740a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 0x7f0a740a57a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a88101aa0 tx=0x7f0a84005f90 comp rx=0 tx=0).stop 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 msgr2=0x7f0a8819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0a80031ae0 tx=0x7f0a80031ee0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 shutdown_connections 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0a740a32e0 0x7f0a740a57a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a88102600 0x7f0a8819f140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a88101c40 0x7f0a8819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 --2- 172.21.15.28:0/3629012360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a88100a50 0x7f0a8806aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 >> 172.21.15.28:0/3629012360 conn(0x7f0a880fc190 msgr2=0x7f0a88104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 shutdown_connections 2024-09-19T09:32:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.213+0000 7f0a8fe7b640 1 -- 172.21.15.28:0/3629012360 wait complete. 2024-09-19T09:32:32.224 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-19T09:32:31.422533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665106,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":682,"num_write":445,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366048,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575776,"statfs":{"total":1151940427776,"available":1151565594624,"internally_reserved":0,"allocated":12943360,"data_stored":7155188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003456"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.321871+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:29.321871+0000","last_peered":"2024-09-19T09:32:29.321871+0000","last_clean":"2024-09-19T09:32:29.321871+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:29.321871+0000","last_undegraded":"2024-09-19T09:32:29.321871+0000","last_fullsized":"2024-09-19T09:32:29.321871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":218,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.335309+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:28.335309+0000","last_peered":"2024-09-19T09:32:28.335309+0000","last_clean":"2024-09-19T09:32:28.335309+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:28.335309+0000","last_undegraded":"2024-09-19T09:32:28.335309+0000","last_fullsized":"2024-09-19T09:32:28.335309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.336164+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:28.336164+0000","last_peered":"2024-09-19T09:32:28.336164+0000","last_clean":"2024-09-19T09:32:28.336164+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:28.336164+0000","last_undegraded":"2024-09-19T09:32:28.336164+0000","last_fullsized":"2024-09-19T09:32:28.336164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66087,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":159,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.313012+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:25.313012+0000","last_peered":"2024-09-19T09:32:25.313012+0000","last_clean":"2024-09-19T09:32:25.313012+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:25.313012+0000","last_undegraded":"2024-09-19T09:32:25.313012+0000","last_fullsized":"2024-09-19T09:32:25.313012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.337569+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:29.337569+0000","last_peered":"2024-09-19T09:32:29.337569+0000","last_clean":"2024-09-19T09:32:29.337569+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:29.337569+0000","last_undegraded":"2024-09-19T09:32:29.337569+0000","last_fullsized":"2024-09-19T09:32:29.337569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313790+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:26.313790+0000","last_peered":"2024-09-19T09:32:26.313790+0000","last_clean":"2024-09-19T09:32:26.313790+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:26.313790+0000","last_undegraded":"2024-09-19T09:32:26.313790+0000","last_fullsized":"2024-09-19T09:32:26.313790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.312856+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:25.312856+0000","last_peered":"2024-09-19T09:32:25.312856+0000","last_clean":"2024-09-19T09:32:25.312856+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:25.312856+0000","last_undegraded":"2024-09-19T09:32:25.312856+0000","last_fullsized":"2024-09-19T09:32:25.312856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'16","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.571339+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:17.571339+0000","last_peered":"2024-09-19T09:32:17.571339+0000","last_clean":"2024-09-19T09:32:17.571339+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:17.571339+0000","last_undegraded":"2024-09-19T09:32:17.571339+0000","last_fullsized":"2024-09-19T09:32:17.571339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372395+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372395+0000","last_peered":"2024-09-19T09:31:49.372395+0000","last_clean":"2024-09-19T09:31:49.372395+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372395+0000","last_undegraded":"2024-09-19T09:31:49.372395+0000","last_fullsized":"2024-09-19T09:31:49.372395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313677+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:26.313677+0000","last_peered":"2024-09-19T09:32:26.313677+0000","last_clean":"2024-09-19T09:32:26.313677+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:26.313677+0000","last_undegraded":"2024-09-19T09:32:26.313677+0000","last_fullsized":"2024-09-19T09:32:26.313677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74504,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":994,"num_read_kb":573,"num_write":262,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1084374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":499976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":493597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:32.229 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:32:32.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.367+0000 7fe954d4b640 1 -- 172.21.15.28:0/1358734350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe950104f00 msgr2=0x7fe9501072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.367+0000 7fe954d4b640 1 --2- 172.21.15.28:0/1358734350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe950104f00 0x7fe9501072f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe944009900 tx=0x7fe94402a410 comp rx=0 tx=0).stop 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 -- 172.21.15.28:0/1358734350 shutdown_connections 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 --2- 172.21.15.28:0/1358734350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 0x7fe950109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 --2- 172.21.15.28:0/1358734350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe950104f00 0x7fe9501072f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 --2- 172.21.15.28:0/1358734350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9501025d0 0x7fe9501049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 -- 172.21.15.28:0/1358734350 >> 172.21.15.28:0/1358734350 conn(0x7fe9500fc190 msgr2=0x7fe9500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 -- 172.21.15.28:0/1358734350 shutdown_connections 2024-09-19T09:32:32.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 -- 172.21.15.28:0/1358734350 wait complete. 2024-09-19T09:32:32.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 Processor -- start 2024-09-19T09:32:32.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.368+0000 7fe954d4b640 1 -- start start 2024-09-19T09:32:32.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 0x7fe950100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 0x7fe950100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010d810 con 0x7fe9501025d0 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010d690 con 0x7fe950104f00 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010d990 con 0x7fe950107830 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55468/0 (socket says 172.21.15.28:55468) 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 -- 172.21.15.28:0/2094074052 learned_addr learned my addr 172.21.15.28:0/2094074052 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94dd9b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 0x7fe950100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94ed9d640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 0x7fe950100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 -- 172.21.15.28:0/2094074052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 msgr2=0x7fe950100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 0x7fe950100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 -- 172.21.15.28:0/2094074052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 msgr2=0x7fe950100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 0x7fe950100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 -- 172.21.15.28:0/2094074052 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe950101350 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94dd9b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 0x7fe950100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe94e59c640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe94000db80 tx=0x7fe94000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940017070 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe940003b70 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9501a1fe0 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9400106c0 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.369+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501a23e0 con 0x7fe9501025d0 2024-09-19T09:32:32.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.370+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9500fdc80 con 0x7fe9501025d0 2024-09-19T09:32:32.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.372+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fe940004020 con 0x7fe9501025d0 2024-09-19T09:32:32.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.372+0000 7fe93f7fe640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 0x7fe9380a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.372+0000 7fe94dd9b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 0x7fe9380a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.372+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe9400cf860 con 0x7fe9501025d0 2024-09-19T09:32:32.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.373+0000 7fe94dd9b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 0x7fe9380a57a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe944006010 tx=0x7fe944032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.376+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe940020070 con 0x7fe9501025d0 2024-09-19T09:32:32.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.549+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe950062630 con 0x7fe9380a32e0 2024-09-19T09:32:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.555+0000 7fe93f7fe640 1 -- 172.21.15.28:0/2094074052 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7fe950062630 con 0x7fe9380a32e0 2024-09-19T09:32:32.559 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 msgr2=0x7fe9380a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 0x7fe9380a57a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe944006010 tx=0x7fe944032cf0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 msgr2=0x7fe950101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe94000db80 tx=0x7fe94000b630 comp rx=0 tx=0).stop 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 shutdown_connections 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9380a32e0 0x7fe9380a57a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe950104f00 0x7fe950100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe950107830 0x7fe950100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 --2- 172.21.15.28:0/2094074052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9501025d0 0x7fe950101f40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.558+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 >> 172.21.15.28:0/2094074052 conn(0x7fe9500fc190 msgr2=0x7fe950102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.559+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 shutdown_connections 2024-09-19T09:32:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.559+0000 7fe954d4b640 1 -- 172.21.15.28:0/2094074052 wait complete. 2024-09-19T09:32:32.570 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-19T09:32:31.422533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665106,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":682,"num_write":445,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366048,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575776,"statfs":{"total":1151940427776,"available":1151565594624,"internally_reserved":0,"allocated":12943360,"data_stored":7155188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003456"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.321871+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:29.321871+0000","last_peered":"2024-09-19T09:32:29.321871+0000","last_clean":"2024-09-19T09:32:29.321871+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:29.321871+0000","last_undegraded":"2024-09-19T09:32:29.321871+0000","last_fullsized":"2024-09-19T09:32:29.321871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":218,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.335309+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:28.335309+0000","last_peered":"2024-09-19T09:32:28.335309+0000","last_clean":"2024-09-19T09:32:28.335309+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:28.335309+0000","last_undegraded":"2024-09-19T09:32:28.335309+0000","last_fullsized":"2024-09-19T09:32:28.335309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.336164+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:28.336164+0000","last_peered":"2024-09-19T09:32:28.336164+0000","last_clean":"2024-09-19T09:32:28.336164+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:28.336164+0000","last_undegraded":"2024-09-19T09:32:28.336164+0000","last_fullsized":"2024-09-19T09:32:28.336164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66087,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":159,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.313012+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:25.313012+0000","last_peered":"2024-09-19T09:32:25.313012+0000","last_clean":"2024-09-19T09:32:25.313012+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:25.313012+0000","last_undegraded":"2024-09-19T09:32:25.313012+0000","last_fullsized":"2024-09-19T09:32:25.313012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.337569+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:29.337569+0000","last_peered":"2024-09-19T09:32:29.337569+0000","last_clean":"2024-09-19T09:32:29.337569+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:29.337569+0000","last_undegraded":"2024-09-19T09:32:29.337569+0000","last_fullsized":"2024-09-19T09:32:29.337569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313790+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:26.313790+0000","last_peered":"2024-09-19T09:32:26.313790+0000","last_clean":"2024-09-19T09:32:26.313790+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:26.313790+0000","last_undegraded":"2024-09-19T09:32:26.313790+0000","last_fullsized":"2024-09-19T09:32:26.313790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.312856+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:25.312856+0000","last_peered":"2024-09-19T09:32:25.312856+0000","last_clean":"2024-09-19T09:32:25.312856+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:25.312856+0000","last_undegraded":"2024-09-19T09:32:25.312856+0000","last_fullsized":"2024-09-19T09:32:25.312856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'16","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.571339+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:17.571339+0000","last_peered":"2024-09-19T09:32:17.571339+0000","last_clean":"2024-09-19T09:32:17.571339+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:17.571339+0000","last_undegraded":"2024-09-19T09:32:17.571339+0000","last_fullsized":"2024-09-19T09:32:17.571339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372395+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372395+0000","last_peered":"2024-09-19T09:31:49.372395+0000","last_clean":"2024-09-19T09:31:49.372395+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372395+0000","last_undegraded":"2024-09-19T09:31:49.372395+0000","last_fullsized":"2024-09-19T09:31:49.372395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313677+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:26.313677+0000","last_peered":"2024-09-19T09:32:26.313677+0000","last_clean":"2024-09-19T09:32:26.313677+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:26.313677+0000","last_undegraded":"2024-09-19T09:32:26.313677+0000","last_fullsized":"2024-09-19T09:32:26.313677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74504,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":994,"num_read_kb":573,"num_write":262,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1084374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":499976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":493597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:32.574 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T09:32:32.575 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 -- 172.21.15.28:0/321508415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d4107830 msgr2=0x7f24d4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 --2- 172.21.15.28:0/321508415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d4107830 0x7f24d4109ca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ca0 tx=0x7f24cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 -- 172.21.15.28:0/321508415 shutdown_connections 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 --2- 172.21.15.28:0/321508415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d4107830 0x7f24d4109ca0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 --2- 172.21.15.28:0/321508415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4104f00 0x7f24d41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 --2- 172.21.15.28:0/321508415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d41025d0 0x7f24d41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 -- 172.21.15.28:0/321508415 >> 172.21.15.28:0/321508415 conn(0x7f24d40fc190 msgr2=0x7f24d40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 -- 172.21.15.28:0/321508415 shutdown_connections 2024-09-19T09:32:32.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.713+0000 7f24db2c9640 1 -- 172.21.15.28:0/321508415 wait complete. 2024-09-19T09:32:32.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 Processor -- start 2024-09-19T09:32:32.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 -- start start 2024-09-19T09:32:32.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d4104f00 0x7f24d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4107830 0x7f24d419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d7f0 con 0x7f24d41025d0 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d670 con 0x7f24d4104f00 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24db2c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d970 con 0x7f24d4107830 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24d9065640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24d9065640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55498/0 (socket says 172.21.15.28:55498) 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24d9065640 1 -- 172.21.15.28:0/414101230 learned_addr learned my addr 172.21.15.28:0/414101230 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24d8864640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d4104f00 0x7f24d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.714+0000 7f24d9866640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4107830 0x7f24d419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 -- 172.21.15.28:0/414101230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4107830 msgr2=0x7f24d419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4107830 0x7f24d419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 -- 172.21.15.28:0/414101230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d4104f00 msgr2=0x7f24d41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d4104f00 0x7f24d41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 -- 172.21.15.28:0/414101230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419fa80 con 0x7f24d41025d0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24d9065640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e9f0 tx=0x7f24d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d000cd80 con 0x7f24d41025d0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d0010910 con 0x7f24d41025d0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419fd70 con 0x7f24d41025d0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0018990 con 0x7f24d41025d0 2024-09-19T09:32:32.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.715+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a0230 con 0x7f24d41025d0 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.716+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d40fdc80 con 0x7f24d41025d0 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.717+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f24d0010ac0 con 0x7f24d41025d0 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.718+0000 7f24ca7fc640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 0x7f24c00a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.718+0000 7f24d8864640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 0x7f24c00a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.718+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f24d00cfce0 con 0x7f24d41025d0 2024-09-19T09:32:32.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.719+0000 7f24d8864640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 0x7f24c00a5820 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24d4199370 tx=0x7f24c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:32.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.723+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f24d009c280 con 0x7f24d41025d0 2024-09-19T09:32:32.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.897+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24d4062630 con 0x7f24c00a3360 2024-09-19T09:32:32.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.903+0000 7f24ca7fc640 1 -- 172.21.15.28:0/414101230 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7f24d4062630 con 0x7f24c00a3360 2024-09-19T09:32:32.906 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:32.908 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:32.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 msgr2=0x7f24c00a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 0x7f24c00a5820 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24d4199370 tx=0x7f24c400b390 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 msgr2=0x7f24d4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e9f0 tx=0x7f24d000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 shutdown_connections 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f24c00a3360 0x7f24c00a5820 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24d4104f00 0x7f24d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24d4107830 0x7f24d419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 --2- 172.21.15.28:0/414101230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24d41025d0 0x7f24d4197e60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 >> 172.21.15.28:0/414101230 conn(0x7f24d40fc190 msgr2=0x7f24d4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 shutdown_connections 2024-09-19T09:32:32.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:32.906+0000 7f24db2c9640 1 -- 172.21.15.28:0/414101230 wait complete. 2024-09-19T09:32:32.916 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:32 smithi028 ceph-mon[30972]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:32:32.917 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:32 smithi028 ceph-mon[30972]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:32.917 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-19T09:32:31.422533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665106,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":682,"num_write":445,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366048,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575776,"statfs":{"total":1151940427776,"available":1151565594624,"internally_reserved":0,"allocated":12943360,"data_stored":7155188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003456"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.321871+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:29.321871+0000","last_peered":"2024-09-19T09:32:29.321871+0000","last_clean":"2024-09-19T09:32:29.321871+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:29.321871+0000","last_undegraded":"2024-09-19T09:32:29.321871+0000","last_fullsized":"2024-09-19T09:32:29.321871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":218,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.335309+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:28.335309+0000","last_peered":"2024-09-19T09:32:28.335309+0000","last_clean":"2024-09-19T09:32:28.335309+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:28.335309+0000","last_undegraded":"2024-09-19T09:32:28.335309+0000","last_fullsized":"2024-09-19T09:32:28.335309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:28.336164+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:28.336164+0000","last_peered":"2024-09-19T09:32:28.336164+0000","last_clean":"2024-09-19T09:32:28.336164+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:28.336164+0000","last_undegraded":"2024-09-19T09:32:28.336164+0000","last_fullsized":"2024-09-19T09:32:28.336164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66087,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":159,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.313012+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:25.313012+0000","last_peered":"2024-09-19T09:32:25.313012+0000","last_clean":"2024-09-19T09:32:25.313012+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:25.313012+0000","last_undegraded":"2024-09-19T09:32:25.313012+0000","last_fullsized":"2024-09-19T09:32:25.313012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.337569+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:29.337569+0000","last_peered":"2024-09-19T09:32:29.337569+0000","last_clean":"2024-09-19T09:32:29.337569+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:29.337569+0000","last_undegraded":"2024-09-19T09:32:29.337569+0000","last_fullsized":"2024-09-19T09:32:29.337569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313790+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:26.313790+0000","last_peered":"2024-09-19T09:32:26.313790+0000","last_clean":"2024-09-19T09:32:26.313790+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:26.313790+0000","last_undegraded":"2024-09-19T09:32:26.313790+0000","last_fullsized":"2024-09-19T09:32:26.313790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:25.312856+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:25.312856+0000","last_peered":"2024-09-19T09:32:25.312856+0000","last_clean":"2024-09-19T09:32:25.312856+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:25.312856+0000","last_undegraded":"2024-09-19T09:32:25.312856+0000","last_fullsized":"2024-09-19T09:32:25.312856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'16","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.571339+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:17.571339+0000","last_peered":"2024-09-19T09:32:17.571339+0000","last_clean":"2024-09-19T09:32:17.571339+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:17.571339+0000","last_undegraded":"2024-09-19T09:32:17.571339+0000","last_fullsized":"2024-09-19T09:32:17.571339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372395+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372395+0000","last_peered":"2024-09-19T09:31:49.372395+0000","last_clean":"2024-09-19T09:31:49.372395+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372395+0000","last_undegraded":"2024-09-19T09:31:49.372395+0000","last_fullsized":"2024-09-19T09:31:49.372395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:26.313677+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:26.313677+0000","last_peered":"2024-09-19T09:32:26.313677+0000","last_clean":"2024-09-19T09:32:26.313677+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:26.313677+0000","last_undegraded":"2024-09-19T09:32:26.313677+0000","last_fullsized":"2024-09-19T09:32:26.313677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74504,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":994,"num_read_kb":573,"num_write":262,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1084374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":499976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":493597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:32.922 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:32 smithi038 ceph-mon[35271]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:32:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:32 smithi038 ceph-mon[35271]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:33.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:32 smithi078 ceph-mon[35619]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:32:33.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:32 smithi078 ceph-mon[35619]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:33.210 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 -- 172.21.15.28:0/2286738308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 msgr2=0x7f3ce810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 --2- 172.21.15.28:0/2286738308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce810da40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009ca0 tx=0x7f3ce002a950 comp rx=0 tx=0).stop 2024-09-19T09:32:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 -- 172.21.15.28:0/2286738308 shutdown_connections 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 --2- 172.21.15.28:0/2286738308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 0x7f3ce81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 --2- 172.21.15.28:0/2286738308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce810da40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 --2- 172.21.15.28:0/2286738308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 0x7f3ce8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 -- 172.21.15.28:0/2286738308 >> 172.21.15.28:0/2286738308 conn(0x7f3ce80fc7d0 msgr2=0x7f3ce80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 -- 172.21.15.28:0/2286738308 shutdown_connections 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.658+0000 7f3cecf68640 1 -- 172.21.15.28:0/2286738308 wait complete. 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 Processor -- start 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 -- start start 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 0x7f3ce80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 0x7f3ce80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:33.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8110fd0 con 0x7f3ce8102ef0 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8110e50 con 0x7f3ce8102570 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3cecf68640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8111150 con 0x7f3ce8100e20 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3ce6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3ce6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37870/0 (socket says 172.21.15.28:37870) 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3ce6d9d640 1 -- 172.21.15.28:0/3656795017 learned_addr learned my addr 172.21.15.28:0/3656795017 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3ce5d9b640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 0x7f3ce80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.659+0000 7f3ce659c640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 0x7f3ce80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 -- 172.21.15.28:0/3656795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 msgr2=0x7f3ce80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 0x7f3ce80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 -- 172.21.15.28:0/3656795017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 msgr2=0x7f3ce80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 0x7f3ce80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 -- 172.21.15.28:0/3656795017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce81a6970 con 0x7f3ce8102ef0 2024-09-19T09:32:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3ce6d9d640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009dc0 tx=0x7f3ce0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0034040 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce002fe10 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce81a6c00 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00047b0 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.660+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce81a7130 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.661+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce8101200 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.663+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f3ce0037020 con 0x7f3ce8102ef0 2024-09-19T09:32:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.663+0000 7f3cd77fe640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 0x7f3cd00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.663+0000 7f3ce659c640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 0x7f3cd00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.663+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3ce00f0780 con 0x7f3ce8102ef0 2024-09-19T09:32:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.664+0000 7f3ce659c640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 0x7f3cd00a5810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005fd0 tx=0x7f3cd800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:33.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.667+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ce00bd480 con 0x7f3ce8102ef0 2024-09-19T09:32:33.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.881+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ce810a0a0 con 0x7f3ce8102ef0 2024-09-19T09:32:33.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.882+0000 7f3cd77fe640 1 -- 172.21.15.28:0/3656795017 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f3ce00bcd80 con 0x7f3ce8102ef0 2024-09-19T09:32:33.883 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:33.883 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:32:33.884 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:32:33.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 msgr2=0x7f3cd00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:33.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 0x7f3cd00a5810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005fd0 tx=0x7f3cd800b390 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 msgr2=0x7f3ce80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009dc0 tx=0x7f3ce0003d00 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 shutdown_connections 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3cd00a3350 0x7f3cd00a5810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ce8102570 0x7f3ce80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ce8102ef0 0x7f3ce80fd700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 --2- 172.21.15.28:0/3656795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ce8100e20 0x7f3ce80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.884+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 >> 172.21.15.28:0/3656795017 conn(0x7f3ce80fc7d0 msgr2=0x7f3ce806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.885+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 shutdown_connections 2024-09-19T09:32:33.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:33.885+0000 7f3cecf68640 1 -- 172.21.15.28:0/3656795017 wait complete. 2024-09-19T09:32:33.895 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:33 smithi028 ceph-mon[30972]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:33.895 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:33 smithi028 ceph-mon[30972]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:33.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:33 smithi038 ceph-mon[35271]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:33.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:33 smithi038 ceph-mon[35271]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:34.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:33 smithi078 ceph-mon[35619]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:34.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:33 smithi078 ceph-mon[35619]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:34.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:32:34.812 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:34.905 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:34 smithi028 ceph-mon[30972]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T09:32:34.905 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:34 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3656795017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:34 smithi038 ceph-mon[35271]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T09:32:34.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:34 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3656795017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:35.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:34 smithi078 ceph-mon[35619]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T09:32:35.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:34 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3656795017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:35.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 -- 172.21.15.28:0/987493456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154100e20 msgr2=0x7f9154101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:35.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 --2- 172.21.15.28:0/987493456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154100e20 0x7f9154101200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9144009900 tx=0x7f914402a480 comp rx=0 tx=0).stop 2024-09-19T09:32:35.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 -- 172.21.15.28:0/987493456 shutdown_connections 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 --2- 172.21.15.28:0/987493456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154102ef0 0x7f915410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 --2- 172.21.15.28:0/987493456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102570 0x7f91541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 --2- 172.21.15.28:0/987493456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154100e20 0x7f9154101200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 -- 172.21.15.28:0/987493456 >> 172.21.15.28:0/987493456 conn(0x7f91540fc7d0 msgr2=0x7f91540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 -- 172.21.15.28:0/987493456 shutdown_connections 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.255+0000 7f915a465640 1 -- 172.21.15.28:0/987493456 wait complete. 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 Processor -- start 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 -- start start 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 0x7f9154199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 0x7f91541a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:35.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9154110f80 con 0x7f9154102570 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9154110e00 con 0x7f9154100e20 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f915a465640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111100 con 0x7f9154102ef0 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37886/0 (socket says 172.21.15.28:37886) 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 -- 172.21.15.28:0/3644558096 learned_addr learned my addr 172.21.15.28:0/3644558096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f9153fff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 0x7f9154199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f9158a02640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 0x7f91541a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 -- 172.21.15.28:0/3644558096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 msgr2=0x7f91541a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 0x7f91541a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 -- 172.21.15.28:0/3644558096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 msgr2=0x7f9154199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 0x7f9154199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f914bfff640 1 -- 172.21.15.28:0/3644558096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91541a4940 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f9158a02640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 0x7f91541a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.256+0000 7f9153fff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 0x7f9154199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f914bfff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d770 tx=0x7f913c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c017070 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f913c005500 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91541a4b10 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c004e00 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.257+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a5020 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.258+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154101200 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.259+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f913c004fa0 con 0x7f9154102570 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.260+0000 7f9151ffb640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 0x7f91400a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.260+0000 7f9153fff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 0x7f91400a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:35.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.260+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f913c0cf530 con 0x7f9154102570 2024-09-19T09:32:35.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.260+0000 7f9153fff640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 0x7f91400a5810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9144005ab0 tx=0x7f9144005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:35.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.264+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f913c09c120 con 0x7f9154102570 2024-09-19T09:32:35.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.444+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9154109090 con 0x7f9154102570 2024-09-19T09:32:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.446+0000 7f9151ffb640 1 -- 172.21.15.28:0/3644558096 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22580 (secure 0 0 0) 0x7f913c09ba20 con 0x7f9154102570 2024-09-19T09:32:35.447 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:35.447 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:31:49.348824+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:32:35.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 msgr2=0x7f91400a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:35.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 0x7f91400a5810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9144005ab0 tx=0x7f9144005a20 comp rx=0 tx=0).stop 2024-09-19T09:32:35.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 msgr2=0x7f91541998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:35.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d770 tx=0x7f913c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 shutdown_connections 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f91400a3350 0x7f91400a5810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9154102ef0 0x7f91541a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9154102570 0x7f91541998c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 --2- 172.21.15.28:0/3644558096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9154100e20 0x7f9154199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.448+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 >> 172.21.15.28:0/3644558096 conn(0x7f91540fc7d0 msgr2=0x7f91540fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.449+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 shutdown_connections 2024-09-19T09:32:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:35.449+0000 7f915a465640 1 -- 172.21.15.28:0/3644558096 wait complete. 2024-09-19T09:32:35.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:35 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3644558096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:35.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:35 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3644558096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:36.032 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T09:32:36.033 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T09:32:36.043 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:32:36.043 DEBUG:teuthology.orchestra.run.smithi028:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T09:32:36.044 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T09:32:36.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:32:36.044 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:32:36.046 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T09:32:36.059 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:32:36.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:32:36.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:35 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3644558096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:36.315 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:36.320 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:36.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:36 smithi028 ceph-mon[30972]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:36.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:36 smithi038 ceph-mon[35271]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:36.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2896710211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 msgr2=0x7fe4c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2896710211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c4103120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009ca0 tx=0x7fe4b402a950 comp rx=0 tx=0).stop 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2896710211 shutdown_connections 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2896710211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 0x7fe4c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2896710211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 0x7fe4c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2896710211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c4103120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.963+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2896710211 >> 172.21.15.28:0/2896710211 conn(0x7fe4c40fc7a0 msgr2=0x7fe4c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.964+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2896710211 shutdown_connections 2024-09-19T09:32:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.964+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2896710211 wait complete. 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.964+0000 7fe4c9e3b640 1 Processor -- start 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 -- start start 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 0x7fe4c4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 0x7fe4c4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4111f70 con 0x7fe4c4102d40 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4111df0 con 0x7fe4c4103660 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c9e3b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41120f0 con 0x7fe4c410c170 2024-09-19T09:32:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37914/0 (socket says 172.21.15.28:37914) 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 -- 172.21.15.28:0/2193615284 learned_addr learned my addr 172.21.15.28:0/2193615284 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 0x7fe4c4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 0x7fe4c4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 -- 172.21.15.28:0/2193615284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 msgr2=0x7fe4c4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 0x7fe4c4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 -- 172.21.15.28:0/2193615284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 msgr2=0x7fe4c4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 0x7fe4c4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.965+0000 7fe4c37fe640 1 -- 172.21.15.28:0/2193615284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41a6930 con 0x7fe4c4102d40 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4005cc0 tx=0x7fe4b402f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c0ff9640 1 -- 172.21.15.28:0/2193615284 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4034040 con 0x7fe4c4102d40 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c0ff9640 1 -- 172.21.15.28:0/2193615284 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b4030920 con 0x7fe4c4102d40 2024-09-19T09:32:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41a6bc0 con 0x7fe4c4102d40 2024-09-19T09:32:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c0ff9640 1 -- 172.21.15.28:0/2193615284 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4004830 con 0x7fe4c4102d40 2024-09-19T09:32:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.966+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a70a0 con 0x7fe4c4102d40 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.968+0000 7fe4c0ff9640 1 -- 172.21.15.28:0/2193615284 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fe4b4037020 con 0x7fe4c4102d40 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 Processor -- start 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 -- start start 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c0ff9640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 0x7fe4ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x56013483ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x56013484d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5601347a5280 con 0x56013484ad40 2024-09-19T09:32:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56013474cb10 con 0x56013483fb60 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4cc214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5601347a1240 con 0x560134840480 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c0ff9640 1 -- 172.21.15.28:0/2193615284 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe4b40f0870 con 0x7fe4c4102d40 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:57634/0 (socket says 172.21.15.28:57634) 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/1054091706 learned_addr learned my addr 172.21.15.28:0/1054091706 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x56013484d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x56013483ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.969+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 0x7fe4ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/1054091706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 msgr2=0x56013483ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x56013483ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/1054091706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 msgr2=0x56013484d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x56013484d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/1054091706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601347591f0 con 0x560134840480 2024-09-19T09:32:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 0x7fe4ac0a5790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x56013483d470 tx=0x7fe4bc005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800bca0 tx=0x7fe4b80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae37ee917b422a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4a9ffb640 1 -- 172.21.15.28:0/1054091706 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8004030 con 0x560134840480 2024-09-19T09:32:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4a9ffb640 1 -- 172.21.15.28:0/1054091706 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80041d0 con 0x560134840480 2024-09-19T09:32:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.970+0000 7fe4a9ffb640 1 -- 172.21.15.28:0/1054091706 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8011770 con 0x560134840480 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- 172.21.15.28:0/1054091706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 msgr2=0x56013484a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800bca0 tx=0x7fe4b80107a0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- 172.21.15.28:0/1054091706 shutdown_connections 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x56013484d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x56013484a800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 --2- 172.21.15.28:0/1054091706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x56013483ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- 172.21.15.28:0/1054091706 >> 172.21.15.28:0/1054091706 conn(0x56013483c0f0 msgr2=0x56013483c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- 172.21.15.28:0/1054091706 shutdown_connections 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- 172.21.15.28:0/1054091706 wait complete. 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 -- start start 2024-09-19T09:32:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.971+0000 7fe4cc214740 1 Processor -- start 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 0x560134847b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56013484d910 con 0x56013484ad40 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56013475ff10 con 0x560134840480 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56013484da90 con 0x56013483fb60 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4cc214740 10 client.0 _ll_register_callbacks cb 0x560134728a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37928/0 (socket says 172.21.15.28:37928) 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 -- 172.21.15.28:0/670816029 learned_addr learned my addr 172.21.15.28:0/670816029 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 0x560134847b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 -- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 msgr2=0x56013484a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 -- 172.21.15.28:0/670816029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 msgr2=0x560134847b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 0x560134847b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c3fff640 1 -- 172.21.15.28:0/670816029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601345166a0 con 0x56013484ad40 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.972+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 0x560134847b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004300 tx=0x7fe4bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98b90f88d8ae370 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc026070 con 0x56013484ad40 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe4bc005020 con 0x56013484ad40 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b80041d0 con 0x56013484ad40 2024-09-19T09:32:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc031050 con 0x56013484ad40 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601348594a0 con 0x56013484ad40 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80041d0 con 0x56013484ad40 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4bc0035f0 con 0x56013484ad40 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 20 client.15312 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56013485a8b0 0x56013485cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 4 client.15312 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.973+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601347b27f0 con 0x56013485a8b0 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.974+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56013485a8b0 0x56013485cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.974+0000 7fe4a2ffd640 10 client.15312 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.974+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56013485a8b0 0x56013485cd50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c06a327ff68fff6 server_cookie=e06ff7532bf0af58 in_seq=0 out_seq=0 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/670816029 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601347b27f0 con 0x56013485a8b0 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4a2ffd640 10 client.15312 handle_command_reply: tid=0 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 client.15312 shutdown 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 msgr2=0x5601348482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004300 tx=0x7fe4bc004730 comp rx=0 tx=0).stop 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 shutdown_connections 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560134840480 0x560134847b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56013485a8b0 0x56013485cd50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x5601348482e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 --2- 172.21.15.28:0/670816029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 >> 172.21.15.28:0/670816029 conn(0x56013483c0f0 msgr2=0x56013483b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4a2ffd640 0 client.15312 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 shutdown_connections 2024-09-19T09:32:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.975+0000 7fe4cc214740 1 -- 172.21.15.28:0/670816029 wait complete. 2024-09-19T09:32:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.976+0000 7fe4cc214740 20 client.15312 trim_cache size 0 max 16384 2024-09-19T09:32:36.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 Processor -- start 2024-09-19T09:32:36.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 -- start start 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x5601348425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x5601348499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41af620 con 0x56013484ad40 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b1160 con 0x56013483fb60 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4cc214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41afcf0 con 0x560134840480 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37942/0 (socket says 172.21.15.28:37942) 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4c3fff640 1 -- 172.21.15.28:0/3037570976 learned_addr learned my addr 172.21.15.28:0/3037570976 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x5601348499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.993+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x5601348425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 -- 172.21.15.28:0/3037570976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 msgr2=0x5601348499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x5601348499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 -- 172.21.15.28:0/3037570976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 msgr2=0x5601348425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x5601348425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 -- 172.21.15.28:0/3037570976 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56013484ab70 con 0x56013484ad40 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x5601348425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0096e0 tx=0x7fe4bc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71311698d6c28f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4a97fa640 1 -- 172.21.15.28:0/3037570976 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc026070 con 0x56013484ad40 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4a97fa640 1 -- 172.21.15.28:0/3037570976 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc0125d0 con 0x56013484ad40 2024-09-19T09:32:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4a97fa640 1 -- 172.21.15.28:0/3037570976 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003e90 con 0x56013484ad40 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4cc214740 1 -- 172.21.15.28:0/3037570976 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 msgr2=0x560134862670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.994+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0096e0 tx=0x7fe4bc00bab0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- 172.21.15.28:0/3037570976 shutdown_connections 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56013484ad40 0x560134862670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560134840480 0x5601348499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3037570976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013483fb60 0x5601348425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- 172.21.15.28:0/3037570976 >> 172.21.15.28:0/3037570976 conn(0x56013483c0f0 msgr2=0x5601348492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- 172.21.15.28:0/3037570976 shutdown_connections 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- 172.21.15.28:0/3037570976 wait complete. 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- start start 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 Processor -- start 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013484ad40 0x5601348be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b0a90 con 0x560134840480 2024-09-19T09:32:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b03c0 con 0x56013484ad40 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.995+0000 7fe4cc214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41ad620 con 0x56013483fb60 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4cc214740 10 client.0 _ll_register_callbacks cb 0x7fe4bc00be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37948/0 (socket says 172.21.15.28:37948) 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/3238221422 learned_addr learned my addr 172.21.15.28:0/3238221422 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c3fff640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013484ad40 0x5601348be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 msgr2=0x56013484f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/3238221422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013484ad40 msgr2=0x5601348be680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013484ad40 0x5601348be680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 -- 172.21.15.28:0/3238221422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601348becc0 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4c2ffd640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x56013483cf50 tx=0x7fe4b800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b1f66a6c8cd2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b802c060 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe4b8005450 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.996+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc0125d0 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4b8005650 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 10 client.15318 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601348c60b0 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc0125d0 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b80276c0 con 0x560134840480 2024-09-19T09:32:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 20 client.15318 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5601348c64a0 0x5601348c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 4 client.15318 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5601347acd00 con 0x5601348c64a0 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.997+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5601348c64a0 0x5601348c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.998+0000 7fe4c37fe640 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5601348c64a0 0x5601348c88c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1335e88b2394ae89 server_cookie=62f1a8abc32c31cf in_seq=0 out_seq=0 2024-09-19T09:32:37.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.998+0000 7fe4a2ffd640 10 client.15318 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:37.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4a2ffd640 1 -- 172.21.15.28:0/3238221422 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5601347acd00 con 0x5601348c64a0 2024-09-19T09:32:37.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4a2ffd640 10 client.15318 handle_command_reply: tid=0 2024-09-19T09:32:37.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 client.15318 shutdown 2024-09-19T09:32:37.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 msgr2=0x56013484f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x56013483cf50 tx=0x7fe4b800bfa0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 shutdown_connections 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56013484ad40 0x5601348be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5601348c64a0 0x5601348c88c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560134840480 0x56013484f900 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 --2- 172.21.15.28:0/3238221422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56013483fb60 0x56013484f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4a2ffd640 0 client.15318 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 >> 172.21.15.28:0/3238221422 conn(0x56013483c0f0 msgr2=0x56013483b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 shutdown_connections 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 1 -- 172.21.15.28:0/3238221422 wait complete. 2024-09-19T09:32:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:36.999+0000 7fe4cc214740 20 client.15318 trim_cache size 0 max 16384 2024-09-19T09:32:37.006 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:37.006 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2024-09-19T09:32:37.006 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "78a4467d-5df6-4fa8-9738-bf5639f297e2", 2024-09-19T09:32:37.006 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2024-09-19T09:32:37.006 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 msgr2=0x7fe4ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 0x7fe4ac0a5790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x56013483d470 tx=0x7fe4bc005cc0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 msgr2=0x7fe4c41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4005cc0 tx=0x7fe4b402f630 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 shutdown_connections 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe4ac0a32d0 0x7fe4ac0a5790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4c4103660 0x7fe4c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c410c170 0x7fe4c4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 --2- 172.21.15.28:0/2193615284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c4102d40 0x7fe4c41009a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 >> 172.21.15.28:0/2193615284 conn(0x7fe4c40fc7a0 msgr2=0x7fe4c40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 shutdown_connections 2024-09-19T09:32:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.000+0000 7fe4c9e3b640 1 -- 172.21.15.28:0/2193615284 wait complete. 2024-09-19T09:32:37.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:36 smithi078 ceph-mon[35619]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 -- 172.21.15.28:0/827940451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 msgr2=0x7f36f8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 --2- 172.21.15.28:0/827940451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f8103190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009930 tx=0x7f36f402a420 comp rx=0 tx=0).stop 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 -- 172.21.15.28:0/827940451 shutdown_connections 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 --2- 172.21.15.28:0/827940451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f8103760 0x7f36f8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 --2- 172.21.15.28:0/827940451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f81082e0 0x7f36f810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 --2- 172.21.15.28:0/827940451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f8103190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 -- 172.21.15.28:0/827940451 >> 172.21.15.28:0/827940451 conn(0x7f36f80fc7d0 msgr2=0x7f36f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 -- 172.21.15.28:0/827940451 shutdown_connections 2024-09-19T09:32:37.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.144+0000 7f37005d4640 1 -- 172.21.15.28:0/827940451 wait complete. 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 Processor -- start 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 -- start start 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f8103760 0x7f36f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f81082e0 0x7f36f819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8112090 con 0x7f36f8102db0 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8111f10 con 0x7f36f81082e0 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.145+0000 7f37005d4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8112210 con 0x7f36f8103760 2024-09-19T09:32:37.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37952/0 (socket says 172.21.15.28:37952) 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 -- 172.21.15.28:0/3607993515 learned_addr learned my addr 172.21.15.28:0/3607993515 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36feb71640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f81082e0 0x7f36f819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fdb6f640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f8103760 0x7f36f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 -- 172.21.15.28:0/3607993515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f8103760 msgr2=0x7f36f8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f8103760 0x7f36f8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 -- 172.21.15.28:0/3607993515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f81082e0 msgr2=0x7f36f819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f81082e0 0x7f36f819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 -- 172.21.15.28:0/3607993515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f819d570 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36fe370640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004770 tx=0x7f36f4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f403f070 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f4031c20 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f819d800 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.146+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4035030 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.147+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f81a5140 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.147+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f8103190 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.149+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f36f4042070 con 0x7f36f8102db0 2024-09-19T09:32:37.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.149+0000 7f36ef7fe640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 0x7f36e40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.149+0000 7f36fdb6f640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 0x7f36e40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.149+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f36f40f0a40 con 0x7f36f8102db0 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.150+0000 7f36fdb6f640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 0x7f36e40a5810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36f80fee70 tx=0x7f36e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.152+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f36f40bcfe0 con 0x7f36f8102db0 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.327+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36f8062630 con 0x7f36e40a3350 2024-09-19T09:32:37.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.334+0000 7f36ef7fe640 1 -- 172.21.15.28:0/3607993515 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258896 (secure 0 0 0) 0x7f36f8062630 con 0x7f36e40a3350 2024-09-19T09:32:37.479 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 msgr2=0x7f36e40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 0x7f36e40a5810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36f80fee70 tx=0x7f36e800b390 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 msgr2=0x7f36f81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004770 tx=0x7f36f4003b90 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 shutdown_connections 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f36e40a3350 0x7f36e40a5810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36f81082e0 0x7f36f819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f8103760 0x7f36f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 --2- 172.21.15.28:0/3607993515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36f8102db0 0x7f36f81985c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 >> 172.21.15.28:0/3607993515 conn(0x7f36f80fc7d0 msgr2=0x7f36f8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:37.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 shutdown_connections 2024-09-19T09:32:37.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:37.474+0000 7f37005d4640 1 -- 172.21.15.28:0/3607993515 wait complete. 2024-09-19T09:32:37.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:37 smithi038 ceph-mon[35271]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:37.993 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:37 smithi028 ceph-mon[30972]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:38.025 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:32:38.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:37 smithi078 ceph-mon[35619]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:38.713 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:38 smithi028 ceph-mon[30972]: scrub queued for path: / 2024-09-19T09:32:38.713 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:38 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:38.713 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:38 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:32:38.713 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:38 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:38.713 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:38 smithi028 ceph-mon[30972]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:38.886 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-19T09:32:35.423401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668327,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1169,"num_read_kb":718,"num_write":450,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366068,"kb_used_data":12660,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575756,"statfs":{"total":1151940427776,"available":1151565574144,"internally_reserved":0,"allocated":12963840,"data_stored":7176861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9688,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":78,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:29.321871+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:29.321871+0000","last_peered":"2024-09-19T09:32:29.321871+0000","last_clean":"2024-09-19T09:32:29.321871+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:29.321871+0000","last_undegraded":"2024-09-19T09:32:29.321871+0000","last_fullsized":"2024-09-19T09:32:29.321871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:33.326896+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:33.326896+0000","last_peered":"2024-09-19T09:32:33.326896+0000","last_clean":"2024-09-19T09:32:33.326896+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:33.326896+0000","last_undegraded":"2024-09-19T09:32:33.326896+0000","last_fullsized":"2024-09-19T09:32:33.326896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:33.343213+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:33.343213+0000","last_peered":"2024-09-19T09:32:33.343213+0000","last_clean":"2024-09-19T09:32:33.343213+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:33.343213+0000","last_undegraded":"2024-09-19T09:32:33.343213+0000","last_fullsized":"2024-09-19T09:32:33.343213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":173,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'9","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:30.320303+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:30.320303+0000","last_peered":"2024-09-19T09:32:30.320303+0000","last_clean":"2024-09-19T09:32:30.320303+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:30.320303+0000","last_undegraded":"2024-09-19T09:32:30.320303+0000","last_fullsized":"2024-09-19T09:32:30.320303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:33.343598+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:33.343598+0000","last_peered":"2024-09-19T09:32:33.343598+0000","last_clean":"2024-09-19T09:32:33.343598+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:33.343598+0000","last_undegraded":"2024-09-19T09:32:33.343598+0000","last_fullsized":"2024-09-19T09:32:33.343598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":128,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:31.321097+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:31.321097+0000","last_peered":"2024-09-19T09:32:31.321097+0000","last_clean":"2024-09-19T09:32:31.321097+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:31.321097+0000","last_undegraded":"2024-09-19T09:32:31.321097+0000","last_fullsized":"2024-09-19T09:32:31.321097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:30.319702+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:30.319702+0000","last_peered":"2024-09-19T09:32:30.319702+0000","last_clean":"2024-09-19T09:32:30.319702+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:30.319702+0000","last_undegraded":"2024-09-19T09:32:30.319702+0000","last_fullsized":"2024-09-19T09:32:30.319702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:27.572366+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:27.572366+0000","last_peered":"2024-09-19T09:32:27.572366+0000","last_clean":"2024-09-19T09:32:27.572366+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:27.572366+0000","last_undegraded":"2024-09-19T09:32:27.572366+0000","last_fullsized":"2024-09-19T09:32:27.572366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372395+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372395+0000","last_peered":"2024-09-19T09:31:49.372395+0000","last_clean":"2024-09-19T09:31:49.372395+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372395+0000","last_undegraded":"2024-09-19T09:31:49.372395+0000","last_fullsized":"2024-09-19T09:31:49.372395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:31.320395+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:31.320395+0000","last_peered":"2024-09-19T09:32:31.320395+0000","last_clean":"2024-09-19T09:32:31.320395+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:31.320395+0000","last_undegraded":"2024-09-19T09:32:31.320395+0000","last_fullsized":"2024-09-19T09:32:31.320395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77725,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1043,"num_read_kb":609,"num_write":267,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1601536,"data_stored":1093053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":502496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":501883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:38.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:32:38.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:38 smithi038 ceph-mon[35271]: scrub queued for path: / 2024-09-19T09:32:38.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:38 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:38 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:32:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:38 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:38 smithi038 ceph-mon[35271]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:38 smithi078 ceph-mon[35619]: scrub queued for path: / 2024-09-19T09:32:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:38 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:38 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:32:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:38 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:32:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:38 smithi078 ceph-mon[35619]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 -- 172.21.15.28:0/696233550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938102ef0 msgr2=0x7f793810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 --2- 172.21.15.28:0/696233550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938102ef0 0x7f793810da40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ca0 tx=0x7f793002a950 comp rx=0 tx=0).stop 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 -- 172.21.15.28:0/696233550 shutdown_connections 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 --2- 172.21.15.28:0/696233550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938102ef0 0x7f793810da40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 --2- 172.21.15.28:0/696233550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102570 0x7f79381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 --2- 172.21.15.28:0/696233550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938100e20 0x7f7938101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 -- 172.21.15.28:0/696233550 >> 172.21.15.28:0/696233550 conn(0x7f79380fc7d0 msgr2=0x7f79380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 -- 172.21.15.28:0/696233550 shutdown_connections 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.334+0000 7f793e760640 1 -- 172.21.15.28:0/696233550 wait complete. 2024-09-19T09:32:39.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 Processor -- start 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 -- start start 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938102570 0x7f79381976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102ef0 0x7f7938197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7938110f80 con 0x7f7938100e20 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7938110e00 con 0x7f7938102570 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793e760640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7938111100 con 0x7f7938102ef0 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f7937fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f7937fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37972/0 (socket says 172.21.15.28:37972) 2024-09-19T09:32:39.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f7937fff640 1 -- 172.21.15.28:0/2775122830 learned_addr learned my addr 172.21.15.28:0/2775122830 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f792ffff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938102570 0x7f79381976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.335+0000 7f793ccfd640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102ef0 0x7f7938197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 -- 172.21.15.28:0/2775122830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102ef0 msgr2=0x7f7938197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102ef0 0x7f7938197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 -- 172.21.15.28:0/2775122830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938102570 msgr2=0x7f79381976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938102570 0x7f79381976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 -- 172.21.15.28:0/2775122830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381a69d0 con 0x7f7938100e20 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7937fff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f792800e9f0 tx=0x7f792800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7935ffb640 1 -- 172.21.15.28:0/2775122830 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792800cd80 con 0x7f7938100e20 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7935ffb640 1 -- 172.21.15.28:0/2775122830 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7928010910 con 0x7f7938100e20 2024-09-19T09:32:39.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381a6cc0 con 0x7f7938100e20 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f7935ffb640 1 -- 172.21.15.28:0/2775122830 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928018990 con 0x7f7938100e20 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.336+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381a71f0 con 0x7f7938100e20 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.338+0000 7f7935ffb640 1 -- 172.21.15.28:0/2775122830 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f7928010ac0 con 0x7f7938100e20 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.338+0000 7f7940b39740 1 Processor -- start 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 -- start start 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c16653140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 0x562c16655a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562c165ae2e0 con 0x562c166484a0 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c16555ce0 con 0x562c16653680 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7940b39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c165aa2a0 con 0x562c16648dc0 2024-09-19T09:32:39.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37976/0 (socket says 172.21.15.28:37976) 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 -- 172.21.15.28:0/558905759 learned_addr learned my addr 172.21.15.28:0/558905759 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7937fff640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c16653140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f792ffff640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 0x562c16655a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 -- 172.21.15.28:0/558905759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 msgr2=0x562c16653140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c16653140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 -- 172.21.15.28:0/558905759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 msgr2=0x562c16655a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 0x562c16655a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f793ccfd640 1 -- 172.21.15.28:0/558905759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1653a440 con 0x562c166484a0 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f792ffff640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 0x562c16655a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:39.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.339+0000 7f7935ffb640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 0x7f79240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f7937fff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 0x7f79240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f793ccfd640 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7930035000 tx=0x7f7930005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f002497e8c4acfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f7935ffb640 1 -- 172.21.15.28:0/2775122830 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f79280cfce0 con 0x7f7938100e20 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f792e7fc640 1 -- 172.21.15.28:0/558905759 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930048070 con 0x562c166484a0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f792e7fc640 1 -- 172.21.15.28:0/558905759 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793002f930 con 0x562c166484a0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.340+0000 7f792e7fc640 1 -- 172.21.15.28:0/558905759 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930034040 con 0x562c166484a0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- 172.21.15.28:0/558905759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 msgr2=0x562c16648880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7937fff640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 0x7f79240a5810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x562c16645890 tx=0x7f7928000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7930035000 tx=0x7f7930005fa0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- 172.21.15.28:0/558905759 shutdown_connections 2024-09-19T09:32:39.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16653680 0x562c16655a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c16653140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 --2- 172.21.15.28:0/558905759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c166484a0 0x562c16648880 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- 172.21.15.28:0/558905759 >> 172.21.15.28:0/558905759 conn(0x562c16644a30 msgr2=0x562c16644e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- 172.21.15.28:0/558905759 shutdown_connections 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- 172.21.15.28:0/558905759 wait complete. 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 -- start start 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.341+0000 7f7940b39740 1 Processor -- start 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16652f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 0x562c16650200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562c16656250 con 0x562c16653680 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c16569050 con 0x562c16648dc0 2024-09-19T09:32:39.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c166563d0 con 0x562c166484a0 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f7940b39740 10 client.0 _ll_register_callbacks cb 0x562c165a9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37978/0 (socket says 172.21.15.28:37978) 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 -- 172.21.15.28:0/3424471808 learned_addr learned my addr 172.21.15.28:0/3424471808 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f792ffff640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 0x562c16650200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16652f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 -- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 msgr2=0x562c16652f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16652f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 -- 172.21.15.28:0/3424471808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 msgr2=0x562c16650200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 0x562c16650200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 -- 172.21.15.28:0/3424471808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1631f3c0 con 0x562c16653680 2024-09-19T09:32:39.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f792ffff640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 0x562c16650200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7930005e30 tx=0x7f7930003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242249c97dac191c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.342+0000 7f79177fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930048070 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7930004270 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793002f930 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7930058070 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 10 client.15336 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c16661b70 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793002f930 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7930030ba0 con 0x562c16653680 2024-09-19T09:32:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.343+0000 7f7940b39740 20 client.15336 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16662ec0 0x562c166652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f7940b39740 4 client.15336 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c165bb850 con 0x562c16662ec0 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f792ffff640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16662ec0 0x562c166652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f792ffff640 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16662ec0 0x562c166652b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b95f9e6183d4935 server_cookie=5b521d19eaa21bc9 in_seq=0 out_seq=0 2024-09-19T09:32:39.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.344+0000 7f79177fe640 10 client.15336 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f79177fe640 1 -- 172.21.15.28:0/3424471808 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c165bb850 con 0x562c16662ec0 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f79177fe640 10 client.15336 handle_command_reply: tid=0 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 client.15336 shutdown 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 msgr2=0x562c16650740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7930005e30 tx=0x7f7930003c00 comp rx=0 tx=0).stop 2024-09-19T09:32:39.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 shutdown_connections 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16648dc0 0x562c16650200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16662ec0 0x562c166652b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f79177fe640 0 client.15336 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16653680 0x562c16650740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 --2- 172.21.15.28:0/3424471808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16652f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 >> 172.21.15.28:0/3424471808 conn(0x562c16644a30 msgr2=0x562c1664a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 shutdown_connections 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 1 -- 172.21.15.28:0/3424471808 wait complete. 2024-09-19T09:32:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.345+0000 7f7940b39740 20 client.15336 trim_cache size 0 max 16384 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 Processor -- start 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 -- start start 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 0x562c16652170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c166682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f79381af5a0 con 0x562c16644a30 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b10e0 con 0x562c166484a0 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f7940b39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79381afc70 con 0x562c16648dc0 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37988/0 (socket says 172.21.15.28:37988) 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f793ccfd640 1 -- 172.21.15.28:0/3837135356 learned_addr learned my addr 172.21.15.28:0/3837135356 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.365+0000 7f792ffff640 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 0x562c16652170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 -- 172.21.15.28:0/3837135356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 msgr2=0x562c166682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c166682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 -- 172.21.15.28:0/3837135356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 msgr2=0x562c16652170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 0x562c16652170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 -- 172.21.15.28:0/3837135356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c166532c0 con 0x562c16644a30 2024-09-19T09:32:39.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f792ffff640 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 0x562c16652170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f793ccfd640 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7930005de0 tx=0x7f793003a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564886b40de66b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f792dffb640 1 -- 172.21.15.28:0/3837135356 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930048040 con 0x562c16644a30 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f792dffb640 1 -- 172.21.15.28:0/3837135356 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7930049040 con 0x562c16644a30 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.366+0000 7f792dffb640 1 -- 172.21.15.28:0/3837135356 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930034040 con 0x562c16644a30 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- 172.21.15.28:0/3837135356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 msgr2=0x562c166501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7930005de0 tx=0x7f793003a730 comp rx=0 tx=0).stop 2024-09-19T09:32:39.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- 172.21.15.28:0/3837135356 shutdown_connections 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c166484a0 0x562c16652170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c16648dc0 0x562c166682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 --2- 172.21.15.28:0/3837135356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16644a30 0x562c166501c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- 172.21.15.28:0/3837135356 >> 172.21.15.28:0/3837135356 conn(0x562c166508a0 msgr2=0x562c16651a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- 172.21.15.28:0/3837135356 shutdown_connections 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- 172.21.15.28:0/3837135356 wait complete. 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.367+0000 7f7940b39740 1 -- start start 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 Processor -- start 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16644a30 0x562c16667df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16668330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b0a10 con 0x562c16648dc0 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b0340 con 0x562c16644a30 2024-09-19T09:32:39.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79381ad580 con 0x562c166484a0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f7940b39740 10 client.0 _ll_register_callbacks cb 0x7f7930046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f792ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16668330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f793ccfd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37994/0 (socket says 172.21.15.28:37994) 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f793ccfd640 1 -- 172.21.15.28:0/2000361585 learned_addr learned my addr 172.21.15.28:0/2000361585 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.368+0000 7f793ccfd640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16644a30 0x562c16667df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 -- 172.21.15.28:0/2000361585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16644a30 msgr2=0x562c16667df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16644a30 0x562c16667df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 -- 172.21.15.28:0/2000361585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 msgr2=0x562c166c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 -- 172.21.15.28:0/2000361585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c166c5130 con 0x562c166484a0 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f793ccfd640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:39.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f792ffff640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16668330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562c16651840 tx=0x7f792000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6239344a2a4ca863 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:39.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f79177fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:32:39.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920023470 con 0x562c166484a0 2024-09-19T09:32:39.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7920023610 con 0x562c166484a0 2024-09-19T09:32:39.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.369+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7930049040 con 0x562c166484a0 2024-09-19T09:32:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7920023810 con 0x562c166484a0 2024-09-19T09:32:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 10 client.25003 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c166cd770 con 0x562c166484a0 2024-09-19T09:32:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7930049040 con 0x562c166484a0 2024-09-19T09:32:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7920011a10 con 0x562c166484a0 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 10 client.25003 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 10 client.25003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 20 client.25003 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16653680 0x562c166cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 4 client.25003 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.370+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c165b5d60 con 0x562c16653680 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f793ccfd640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16653680 0x562c166cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f793ccfd640 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16653680 0x562c166cfee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8132d8da5ddf11e server_cookie=c584e404191b5040 in_seq=0 out_seq=0 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f79177fe640 10 client.25003 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f79177fe640 1 -- 172.21.15.28:0/2000361585 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562c165b5d60 con 0x562c16653680 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f79177fe640 10 client.25003 handle_command_reply: tid=0 2024-09-19T09:32:39.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f7940b39740 1 client.25003 shutdown 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 msgr2=0x562c16668330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.371+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16668330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562c16651840 tx=0x7f792000bdd0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 shutdown_connections 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562c16653680 0x562c166cfee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562c16648dc0 0x562c166c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c166484a0 0x562c16668330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 --2- 172.21.15.28:0/2000361585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c16644a30 0x562c16667df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f79177fe640 0 client.25003 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 >> 172.21.15.28:0/2000361585 conn(0x562c166508a0 msgr2=0x562c16652280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 shutdown_connections 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 1 -- 172.21.15.28:0/2000361585 wait complete. 2024-09-19T09:32:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f7940b39740 20 client.25003 trim_cache size 0 max 16384 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "78a4467d-5df6-4fa8-9738-bf5639f297e2": { 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "78a4467d-5df6-4fa8-9738-bf5639f297e2", 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive,force" 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:32:39.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:32:39.379 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 msgr2=0x7f79240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 0x7f79240a5810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x562c16645890 tx=0x7f7928000bc0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 msgr2=0x7f7938197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.372+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f792800e9f0 tx=0x7f792800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 shutdown_connections 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79240a3350 0x7f79240a5810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7938102570 0x7f79381976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7938102ef0 0x7f7938197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 --2- 172.21.15.28:0/2775122830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7938100e20 0x7f7938197160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 >> 172.21.15.28:0/2775122830 conn(0x7f79380fc7d0 msgr2=0x7f7938106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 shutdown_connections 2024-09-19T09:32:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:39.373+0000 7f793e760640 1 -- 172.21.15.28:0/2775122830 wait complete. 2024-09-19T09:32:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:39.901 INFO:tasks.cephfs.filesystem:scrub status for tag:78a4467d-5df6-4fa8-9738-bf5639f297e2 - {'path': '/', 'tag': '78a4467d-5df6-4fa8-9738-bf5639f297e2', 'options': 'recursive,force'} 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.306+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3173045117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 msgr2=0x7f9f3810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.306+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3173045117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f3810bb50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009900 tx=0x7f9f2c02a480 comp rx=0 tx=0).stop 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3173045117 shutdown_connections 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3173045117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3173045117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f3810bb50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3173045117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 0x7f9f38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3173045117 >> 172.21.15.28:0/3173045117 conn(0x7f9f380fc7a0 msgr2=0x7f9f380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3173045117 shutdown_connections 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3173045117 wait complete. 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.307+0000 7f9f3d0f5640 1 Processor -- start 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 -- start start 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 0x7f9f38198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38111fb0 con 0x7f9f38103660 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38111e30 con 0x7f9f38102d40 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3d0f5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38112130 con 0x7f9f3810c170 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38006/0 (socket says 172.21.15.28:38006) 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 -- 172.21.15.28:0/3609758209 learned_addr learned my addr 172.21.15.28:0/3609758209 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f36d9d640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 0x7f9f38198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3759e640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 -- 172.21.15.28:0/3609758209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 msgr2=0x7f9f3819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 -- 172.21.15.28:0/3609758209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 msgr2=0x7f9f38198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 0x7f9f38198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3659c640 1 -- 172.21.15.28:0/3609758209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3819d3c0 con 0x7f9f38103660 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.308+0000 7f9f3759e640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f3659c640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005ab0 tx=0x7f9f2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f030 con 0x7f9f38103660 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3819d650 con 0x7f9f38103660 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c003ce0 con 0x7f9f38103660 2024-09-19T09:32:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c035070 con 0x7f9f38103660 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.309+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a4f90 con 0x7f9f38103660 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.310+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38103120 con 0x7f9f38103660 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.311+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f9f2c042030 con 0x7f9f38103660 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.312+0000 7f9f1ffff640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.312+0000 7f9f36d9d640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.312+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9f2c0f02e0 con 0x7f9f38103660 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.312+0000 7f9f36d9d640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 0x7f9f200a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800abe0 tx=0x7f9f2800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.316+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f2c0bce70 con 0x7f9f38103660 2024-09-19T09:32:40.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.489+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f380feac0 con 0x7f9f200a32d0 2024-09-19T09:32:40.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.495+0000 7f9f1ffff640 1 -- 172.21.15.28:0/3609758209 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258896 (secure 0 0 0) 0x7f9f380feac0 con 0x7f9f200a32d0 2024-09-19T09:32:40.498 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:40.499 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:40.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 msgr2=0x7f9f200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 0x7f9f200a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800abe0 tx=0x7f9f2800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 msgr2=0x7f9f38198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005ab0 tx=0x7f9f2c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 shutdown_connections 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f3810c170 0x7f9f3819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f38103660 0x7f9f38198950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 --2- 172.21.15.28:0/3609758209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f38102d40 0x7f9f38198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 >> 172.21.15.28:0/3609758209 conn(0x7f9f380fc7a0 msgr2=0x7f9f380fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 shutdown_connections 2024-09-19T09:32:40.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:40.498+0000 7f9f3d0f5640 1 -- 172.21.15.28:0/3609758209 wait complete. 2024-09-19T09:32:40.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:40 smithi038 ceph-mon[35271]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:32:41.002 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:40 smithi028 ceph-mon[30972]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:32:41.092 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-19T09:32:39.424355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668327,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1216,"num_read_kb":745,"num_write":450,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366068,"kb_used_data":12660,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575756,"statfs":{"total":1151940427776,"available":1151565574144,"internally_reserved":0,"allocated":12963840,"data_stored":7176861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9688,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":90,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:34.329004+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:34.329004+0000","last_peered":"2024-09-19T09:32:34.329004+0000","last_clean":"2024-09-19T09:32:34.329004+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:34.329004+0000","last_undegraded":"2024-09-19T09:32:34.329004+0000","last_fullsized":"2024-09-19T09:32:34.329004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":238,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:38.333921+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:38.333921+0000","last_peered":"2024-09-19T09:32:38.333921+0000","last_clean":"2024-09-19T09:32:38.333921+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:38.333921+0000","last_undegraded":"2024-09-19T09:32:38.333921+0000","last_fullsized":"2024-09-19T09:32:38.333921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:38.349877+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:38.349877+0000","last_peered":"2024-09-19T09:32:38.349877+0000","last_clean":"2024-09-19T09:32:38.349877+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:38.349877+0000","last_undegraded":"2024-09-19T09:32:38.349877+0000","last_fullsized":"2024-09-19T09:32:38.349877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":179,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'9","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:35.326398+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:35.326398+0000","last_peered":"2024-09-19T09:32:35.326398+0000","last_clean":"2024-09-19T09:32:35.326398+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:35.326398+0000","last_undegraded":"2024-09-19T09:32:35.326398+0000","last_fullsized":"2024-09-19T09:32:35.326398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:33.343598+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:33.343598+0000","last_peered":"2024-09-19T09:32:33.343598+0000","last_clean":"2024-09-19T09:32:33.343598+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:33.343598+0000","last_undegraded":"2024-09-19T09:32:33.343598+0000","last_fullsized":"2024-09-19T09:32:33.343598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:36.327126+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:36.327126+0000","last_peered":"2024-09-19T09:32:36.327126+0000","last_clean":"2024-09-19T09:32:36.327126+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:36.327126+0000","last_undegraded":"2024-09-19T09:32:36.327126+0000","last_fullsized":"2024-09-19T09:32:36.327126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:35.326317+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:35.326317+0000","last_peered":"2024-09-19T09:32:35.326317+0000","last_clean":"2024-09-19T09:32:35.326317+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:35.326317+0000","last_undegraded":"2024-09-19T09:32:35.326317+0000","last_fullsized":"2024-09-19T09:32:35.326317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:27.572366+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:27.572366+0000","last_peered":"2024-09-19T09:32:27.572366+0000","last_clean":"2024-09-19T09:32:27.572366+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:27.572366+0000","last_undegraded":"2024-09-19T09:32:27.572366+0000","last_fullsized":"2024-09-19T09:32:27.572366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372395+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372395+0000","last_peered":"2024-09-19T09:31:49.372395+0000","last_clean":"2024-09-19T09:31:49.372395+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372395+0000","last_undegraded":"2024-09-19T09:31:49.372395+0000","last_fullsized":"2024-09-19T09:31:49.372395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:36.999102+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:36.999102+0000","last_peered":"2024-09-19T09:32:36.999102+0000","last_clean":"2024-09-19T09:32:36.999102+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:36.999102+0000","last_undegraded":"2024-09-19T09:32:36.999102+0000","last_fullsized":"2024-09-19T09:32:36.999102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77725,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1090,"num_read_kb":636,"num_write":267,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1601536,"data_stored":1093053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":502496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":501883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:41.096 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:32:41.096 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:32:41.106 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:40 smithi078 ceph-mon[35619]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:32:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:41.750 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:41 smithi028 ceph-mon[30972]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:41.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 -- 172.21.15.28:0/387576545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50101de0 msgr2=0x7fbd501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 --2- 172.21.15.28:0/387576545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50101de0 0x7fbd501021c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009900 tx=0x7fbd4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 -- 172.21.15.28:0/387576545 shutdown_connections 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 --2- 172.21.15.28:0/387576545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50106f10 0x7fbd5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 --2- 172.21.15.28:0/387576545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50102790 0x7fbd501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 --2- 172.21.15.28:0/387576545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50101de0 0x7fbd501021c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 -- 172.21.15.28:0/387576545 >> 172.21.15.28:0/387576545 conn(0x7fbd500fd8c0 msgr2=0x7fbd500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 -- 172.21.15.28:0/387576545 shutdown_connections 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.830+0000 7fbd577ca640 1 -- 172.21.15.28:0/387576545 wait complete. 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 Processor -- start 2024-09-19T09:32:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 -- start start 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50101de0 0x7fbd50198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 0x7fbd5019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50111f30 con 0x7fbd50102790 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50111db0 con 0x7fbd50101de0 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd577ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd501120b0 con 0x7fbd50106f10 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd54d65640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd54d65640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38030/0 (socket says 172.21.15.28:38030) 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.831+0000 7fbd54d65640 1 -- 172.21.15.28:0/2871386669 learned_addr learned my addr 172.21.15.28:0/2871386669 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd55566640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50101de0 0x7fbd50198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd55d67640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 0x7fbd5019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 -- 172.21.15.28:0/2871386669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 msgr2=0x7fbd5019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 0x7fbd5019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 -- 172.21.15.28:0/2871386669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50101de0 msgr2=0x7fbd50198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50101de0 0x7fbd50198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 -- 172.21.15.28:0/2871386669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5019d340 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd55d67640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 0x7fbd5019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd54d65640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c920 tx=0x7fbd4000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40007c70 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd40007e10 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5019d630 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40018890 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.832+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd501a4f10 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.833+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd501021c0 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.834+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fbd40018a70 con 0x7fbd50102790 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.835+0000 7fbd467fc640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 0x7fbd3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.835+0000 7fbd55566640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 0x7fbd3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:41.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.835+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbd400cfc40 con 0x7fbd50102790 2024-09-19T09:32:41.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.836+0000 7fbd55566640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 0x7fbd3c0a5810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005ab0 tx=0x7fbd4c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:41.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:41.839+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd400d6080 con 0x7fbd50102790 2024-09-19T09:32:41.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:41 smithi038 ceph-mon[35271]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:42.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.010+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 --> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd50062630 con 0x7fbd3c0a3350 2024-09-19T09:32:42.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.017+0000 7fbd467fc640 1 -- 172.21.15.28:0/2871386669 <== mgr.14150 v2:172.21.15.28:6800/147346946 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7fbd50062630 con 0x7fbd3c0a3350 2024-09-19T09:32:42.021 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-19T09:32:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 msgr2=0x7fbd3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 0x7fbd3c0a5810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005ab0 tx=0x7fbd4c005a20 comp rx=0 tx=0).stop 2024-09-19T09:32:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 msgr2=0x7fbd501988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c920 tx=0x7fbd4000cde0 comp rx=0 tx=0).stop 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 shutdown_connections 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd3c0a3350 0x7fbd3c0a5810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd50106f10 0x7fbd5019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd50102790 0x7fbd501988d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 --2- 172.21.15.28:0/2871386669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd50101de0 0x7fbd50198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 >> 172.21.15.28:0/2871386669 conn(0x7fbd500fd8c0 msgr2=0x7fbd500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 shutdown_connections 2024-09-19T09:32:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:42.021+0000 7fbd577ca640 1 -- 172.21.15.28:0/2871386669 wait complete. 2024-09-19T09:32:42.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:41 smithi078 ceph-mon[35619]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:42.726 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-19T09:32:41.424902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668327,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1226,"num_read_kb":745,"num_write":450,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180965,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366068,"kb_used_data":12660,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575756,"statfs":{"total":1151940427776,"available":1151565574144,"internally_reserved":0,"allocated":12963840,"data_stored":7176861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3614,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":64,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002956"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399710+0000","last_change":"2024-09-19T09:30:42.566771+0000","last_active":"2024-09-19T09:32:17.399710+0000","last_peered":"2024-09-19T09:32:17.399710+0000","last_clean":"2024-09-19T09:32:17.399710+0000","last_became_active":"2024-09-19T09:30:42.566353+0000","last_became_peered":"2024-09-19T09:30:42.566353+0000","last_unstale":"2024-09-19T09:32:17.399710+0000","last_undegraded":"2024-09-19T09:32:17.399710+0000","last_fullsized":"2024-09-19T09:32:17.399710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399960+0000","last_change":"2024-09-19T09:30:42.556198+0000","last_active":"2024-09-19T09:32:17.399960+0000","last_peered":"2024-09-19T09:32:17.399960+0000","last_clean":"2024-09-19T09:32:17.399960+0000","last_became_active":"2024-09-19T09:30:42.555870+0000","last_became_peered":"2024-09-19T09:30:42.555870+0000","last_unstale":"2024-09-19T09:32:17.399960+0000","last_undegraded":"2024-09-19T09:32:17.399960+0000","last_fullsized":"2024-09-19T09:32:17.399960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:12.900517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401736+0000","last_change":"2024-09-19T09:30:42.554880+0000","last_active":"2024-09-19T09:32:17.401736+0000","last_peered":"2024-09-19T09:32:17.401736+0000","last_clean":"2024-09-19T09:32:17.401736+0000","last_became_active":"2024-09-19T09:30:42.554605+0000","last_became_peered":"2024-09-19T09:30:42.554605+0000","last_unstale":"2024-09-19T09:32:17.401736+0000","last_undegraded":"2024-09-19T09:32:17.401736+0000","last_fullsized":"2024-09-19T09:32:17.401736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:13.036977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.385043+0000","last_change":"2024-09-19T09:30:42.571141+0000","last_active":"2024-09-19T09:32:17.385043+0000","last_peered":"2024-09-19T09:32:17.385043+0000","last_clean":"2024-09-19T09:32:17.385043+0000","last_became_active":"2024-09-19T09:30:42.570339+0000","last_became_peered":"2024-09-19T09:30:42.570339+0000","last_unstale":"2024-09-19T09:32:17.385043+0000","last_undegraded":"2024-09-19T09:32:17.385043+0000","last_fullsized":"2024-09-19T09:32:17.385043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:09.653682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380809+0000","last_change":"2024-09-19T09:30:42.551365+0000","last_active":"2024-09-19T09:32:17.380809+0000","last_peered":"2024-09-19T09:32:17.380809+0000","last_clean":"2024-09-19T09:32:17.380809+0000","last_became_active":"2024-09-19T09:30:42.551025+0000","last_became_peered":"2024-09-19T09:30:42.551025+0000","last_unstale":"2024-09-19T09:32:17.380809+0000","last_undegraded":"2024-09-19T09:32:17.380809+0000","last_fullsized":"2024-09-19T09:32:17.380809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:07.021446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388711+0000","last_change":"2024-09-19T09:30:42.564692+0000","last_active":"2024-09-19T09:32:17.388711+0000","last_peered":"2024-09-19T09:32:17.388711+0000","last_clean":"2024-09-19T09:32:17.388711+0000","last_became_active":"2024-09-19T09:30:42.564332+0000","last_became_peered":"2024-09-19T09:30:42.564332+0000","last_unstale":"2024-09-19T09:32:17.388711+0000","last_undegraded":"2024-09-19T09:32:17.388711+0000","last_fullsized":"2024-09-19T09:32:17.388711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:37.384864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393070+0000","last_change":"2024-09-19T09:30:42.558786+0000","last_active":"2024-09-19T09:32:17.393070+0000","last_peered":"2024-09-19T09:32:17.393070+0000","last_clean":"2024-09-19T09:32:17.393070+0000","last_became_active":"2024-09-19T09:30:42.558456+0000","last_became_peered":"2024-09-19T09:30:42.558456+0000","last_unstale":"2024-09-19T09:32:17.393070+0000","last_undegraded":"2024-09-19T09:32:17.393070+0000","last_fullsized":"2024-09-19T09:32:17.393070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:45.003895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393622+0000","last_change":"2024-09-19T09:30:42.561942+0000","last_active":"2024-09-19T09:32:17.393622+0000","last_peered":"2024-09-19T09:32:17.393622+0000","last_clean":"2024-09-19T09:32:17.393622+0000","last_became_active":"2024-09-19T09:30:42.561625+0000","last_became_peered":"2024-09-19T09:30:42.561625+0000","last_unstale":"2024-09-19T09:32:17.393622+0000","last_undegraded":"2024-09-19T09:32:17.393622+0000","last_fullsized":"2024-09-19T09:32:17.393622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:08.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401225+0000","last_change":"2024-09-19T09:30:42.562334+0000","last_active":"2024-09-19T09:32:17.401225+0000","last_peered":"2024-09-19T09:32:17.401225+0000","last_clean":"2024-09-19T09:32:17.401225+0000","last_became_active":"2024-09-19T09:30:42.562012+0000","last_became_peered":"2024-09-19T09:30:42.562012+0000","last_unstale":"2024-09-19T09:32:17.401225+0000","last_undegraded":"2024-09-19T09:32:17.401225+0000","last_fullsized":"2024-09-19T09:32:17.401225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:04.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392281+0000","last_change":"2024-09-19T09:31:49.375182+0000","last_active":"2024-09-19T09:32:17.392281+0000","last_peered":"2024-09-19T09:32:17.392281+0000","last_clean":"2024-09-19T09:32:17.392281+0000","last_became_active":"2024-09-19T09:31:49.374766+0000","last_became_peered":"2024-09-19T09:31:49.374766+0000","last_unstale":"2024-09-19T09:32:17.392281+0000","last_undegraded":"2024-09-19T09:32:17.392281+0000","last_fullsized":"2024-09-19T09:32:17.392281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:10.217709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401411+0000","last_change":"2024-09-19T09:30:42.566739+0000","last_active":"2024-09-19T09:32:17.401411+0000","last_peered":"2024-09-19T09:32:17.401411+0000","last_clean":"2024-09-19T09:32:17.401411+0000","last_became_active":"2024-09-19T09:30:42.564655+0000","last_became_peered":"2024-09-19T09:30:42.564655+0000","last_unstale":"2024-09-19T09:32:17.401411+0000","last_undegraded":"2024-09-19T09:32:17.401411+0000","last_fullsized":"2024-09-19T09:32:17.401411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:14.812610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.380573+0000","last_change":"2024-09-19T09:30:42.570760+0000","last_active":"2024-09-19T09:32:17.380573+0000","last_peered":"2024-09-19T09:32:17.380573+0000","last_clean":"2024-09-19T09:32:17.380573+0000","last_became_active":"2024-09-19T09:30:42.569910+0000","last_became_peered":"2024-09-19T09:30:42.569910+0000","last_unstale":"2024-09-19T09:32:17.380573+0000","last_undegraded":"2024-09-19T09:32:17.380573+0000","last_fullsized":"2024-09-19T09:32:17.380573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.478959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390978+0000","last_change":"2024-09-19T09:30:42.559628+0000","last_active":"2024-09-19T09:32:17.390978+0000","last_peered":"2024-09-19T09:32:17.390978+0000","last_clean":"2024-09-19T09:32:17.390978+0000","last_became_active":"2024-09-19T09:30:42.559327+0000","last_became_peered":"2024-09-19T09:30:42.559327+0000","last_unstale":"2024-09-19T09:32:17.390978+0000","last_undegraded":"2024-09-19T09:32:17.390978+0000","last_fullsized":"2024-09-19T09:32:17.390978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:22.044157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400029+0000","last_change":"2024-09-19T09:30:42.556204+0000","last_active":"2024-09-19T09:32:17.400029+0000","last_peered":"2024-09-19T09:32:17.400029+0000","last_clean":"2024-09-19T09:32:17.400029+0000","last_became_active":"2024-09-19T09:30:42.555806+0000","last_became_peered":"2024-09-19T09:30:42.555806+0000","last_unstale":"2024-09-19T09:32:17.400029+0000","last_undegraded":"2024-09-19T09:32:17.400029+0000","last_fullsized":"2024-09-19T09:32:17.400029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:52.695286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399203+0000","last_change":"2024-09-19T09:30:42.563948+0000","last_active":"2024-09-19T09:32:17.399203+0000","last_peered":"2024-09-19T09:32:17.399203+0000","last_clean":"2024-09-19T09:32:17.399203+0000","last_became_active":"2024-09-19T09:30:42.563635+0000","last_became_peered":"2024-09-19T09:30:42.563635+0000","last_unstale":"2024-09-19T09:32:17.399203+0000","last_undegraded":"2024-09-19T09:32:17.399203+0000","last_fullsized":"2024-09-19T09:32:17.399203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:48.479648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401900+0000","last_change":"2024-09-19T09:30:42.553748+0000","last_active":"2024-09-19T09:32:17.401900+0000","last_peered":"2024-09-19T09:32:17.401900+0000","last_clean":"2024-09-19T09:32:17.401900+0000","last_became_active":"2024-09-19T09:30:42.553429+0000","last_became_peered":"2024-09-19T09:30:42.553429+0000","last_unstale":"2024-09-19T09:32:17.401900+0000","last_undegraded":"2024-09-19T09:32:17.401900+0000","last_fullsized":"2024-09-19T09:32:17.401900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:53.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399849+0000","last_change":"2024-09-19T09:30:42.567102+0000","last_active":"2024-09-19T09:32:17.399849+0000","last_peered":"2024-09-19T09:32:17.399849+0000","last_clean":"2024-09-19T09:32:17.399849+0000","last_became_active":"2024-09-19T09:30:42.566578+0000","last_became_peered":"2024-09-19T09:30:42.566578+0000","last_unstale":"2024-09-19T09:32:17.399849+0000","last_undegraded":"2024-09-19T09:32:17.399849+0000","last_fullsized":"2024-09-19T09:32:17.399849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:37.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.379085+0000","last_change":"2024-09-19T09:30:42.556264+0000","last_active":"2024-09-19T09:32:17.379085+0000","last_peered":"2024-09-19T09:32:17.379085+0000","last_clean":"2024-09-19T09:32:17.379085+0000","last_became_active":"2024-09-19T09:30:42.555951+0000","last_became_peered":"2024-09-19T09:30:42.555951+0000","last_unstale":"2024-09-19T09:32:17.379085+0000","last_undegraded":"2024-09-19T09:32:17.379085+0000","last_fullsized":"2024-09-19T09:32:17.379085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:58.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396179+0000","last_change":"2024-09-19T09:30:42.558846+0000","last_active":"2024-09-19T09:32:17.396179+0000","last_peered":"2024-09-19T09:32:17.396179+0000","last_clean":"2024-09-19T09:32:17.396179+0000","last_became_active":"2024-09-19T09:30:42.558193+0000","last_became_peered":"2024-09-19T09:30:42.558193+0000","last_unstale":"2024-09-19T09:32:17.396179+0000","last_undegraded":"2024-09-19T09:32:17.396179+0000","last_fullsized":"2024-09-19T09:32:17.396179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:46.786083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396117+0000","last_change":"2024-09-19T09:30:42.569139+0000","last_active":"2024-09-19T09:32:17.396117+0000","last_peered":"2024-09-19T09:32:17.396117+0000","last_clean":"2024-09-19T09:32:17.396117+0000","last_became_active":"2024-09-19T09:30:42.568743+0000","last_became_peered":"2024-09-19T09:30:42.568743+0000","last_unstale":"2024-09-19T09:32:17.396117+0000","last_undegraded":"2024-09-19T09:32:17.396117+0000","last_fullsized":"2024-09-19T09:32:17.396117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:41.955482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.389670+0000","last_change":"2024-09-19T09:30:42.558832+0000","last_active":"2024-09-19T09:32:17.389670+0000","last_peered":"2024-09-19T09:32:17.389670+0000","last_clean":"2024-09-19T09:32:17.389670+0000","last_became_active":"2024-09-19T09:30:42.558469+0000","last_became_peered":"2024-09-19T09:30:42.558469+0000","last_unstale":"2024-09-19T09:32:17.389670+0000","last_undegraded":"2024-09-19T09:32:17.389670+0000","last_fullsized":"2024-09-19T09:32:17.389670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:20.129983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400776+0000","last_change":"2024-09-19T09:30:42.558850+0000","last_active":"2024-09-19T09:32:17.400776+0000","last_peered":"2024-09-19T09:32:17.400776+0000","last_clean":"2024-09-19T09:32:17.400776+0000","last_became_active":"2024-09-19T09:30:42.558432+0000","last_became_peered":"2024-09-19T09:30:42.558432+0000","last_unstale":"2024-09-19T09:32:17.400776+0000","last_undegraded":"2024-09-19T09:32:17.400776+0000","last_fullsized":"2024-09-19T09:32:17.400776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.615370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392484+0000","last_change":"2024-09-19T09:30:42.553069+0000","last_active":"2024-09-19T09:32:17.392484+0000","last_peered":"2024-09-19T09:32:17.392484+0000","last_clean":"2024-09-19T09:32:17.392484+0000","last_became_active":"2024-09-19T09:30:42.552621+0000","last_became_peered":"2024-09-19T09:30:42.552621+0000","last_unstale":"2024-09-19T09:32:17.392484+0000","last_undegraded":"2024-09-19T09:32:17.392484+0000","last_fullsized":"2024-09-19T09:32:17.392484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.410131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393139+0000","last_change":"2024-09-19T09:30:42.559429+0000","last_active":"2024-09-19T09:32:17.393139+0000","last_peered":"2024-09-19T09:32:17.393139+0000","last_clean":"2024-09-19T09:32:17.393139+0000","last_became_active":"2024-09-19T09:30:42.559052+0000","last_became_peered":"2024-09-19T09:30:42.559052+0000","last_unstale":"2024-09-19T09:32:17.393139+0000","last_undegraded":"2024-09-19T09:32:17.393139+0000","last_fullsized":"2024-09-19T09:32:17.393139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:51:10.099101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.384058+0000","last_change":"2024-09-19T09:30:42.570537+0000","last_active":"2024-09-19T09:32:17.384058+0000","last_peered":"2024-09-19T09:32:17.384058+0000","last_clean":"2024-09-19T09:32:17.384058+0000","last_became_active":"2024-09-19T09:30:42.570215+0000","last_became_peered":"2024-09-19T09:30:42.570215+0000","last_unstale":"2024-09-19T09:32:17.384058+0000","last_undegraded":"2024-09-19T09:32:17.384058+0000","last_fullsized":"2024-09-19T09:32:17.384058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:46.480769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401151+0000","last_change":"2024-09-19T09:30:42.566740+0000","last_active":"2024-09-19T09:32:17.401151+0000","last_peered":"2024-09-19T09:32:17.401151+0000","last_clean":"2024-09-19T09:32:17.401151+0000","last_became_active":"2024-09-19T09:30:42.566393+0000","last_became_peered":"2024-09-19T09:30:42.566393+0000","last_unstale":"2024-09-19T09:32:17.401151+0000","last_undegraded":"2024-09-19T09:32:17.401151+0000","last_fullsized":"2024-09-19T09:32:17.401151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:31.910821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392078+0000","last_change":"2024-09-19T09:31:49.368470+0000","last_active":"2024-09-19T09:32:17.392078+0000","last_peered":"2024-09-19T09:32:17.392078+0000","last_clean":"2024-09-19T09:32:17.392078+0000","last_became_active":"2024-09-19T09:31:49.368010+0000","last_became_peered":"2024-09-19T09:31:49.368010+0000","last_unstale":"2024-09-19T09:32:17.392078+0000","last_undegraded":"2024-09-19T09:32:17.392078+0000","last_fullsized":"2024-09-19T09:32:17.392078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:31.021903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.387082+0000","last_change":"2024-09-19T09:30:42.562251+0000","last_active":"2024-09-19T09:32:17.387082+0000","last_peered":"2024-09-19T09:32:17.387082+0000","last_clean":"2024-09-19T09:32:17.387082+0000","last_became_active":"2024-09-19T09:30:42.561927+0000","last_became_peered":"2024-09-19T09:30:42.561927+0000","last_unstale":"2024-09-19T09:32:17.387082+0000","last_undegraded":"2024-09-19T09:32:17.387082+0000","last_fullsized":"2024-09-19T09:32:17.387082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:54.998579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399371+0000","last_change":"2024-09-19T09:30:42.563766+0000","last_active":"2024-09-19T09:32:17.399371+0000","last_peered":"2024-09-19T09:32:17.399371+0000","last_clean":"2024-09-19T09:32:17.399371+0000","last_became_active":"2024-09-19T09:30:42.563423+0000","last_became_peered":"2024-09-19T09:30:42.563423+0000","last_unstale":"2024-09-19T09:32:17.399371+0000","last_undegraded":"2024-09-19T09:32:17.399371+0000","last_fullsized":"2024-09-19T09:32:17.399371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:48.665728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393230+0000","last_change":"2024-09-19T09:30:42.552681+0000","last_active":"2024-09-19T09:32:17.393230+0000","last_peered":"2024-09-19T09:32:17.393230+0000","last_clean":"2024-09-19T09:32:17.393230+0000","last_became_active":"2024-09-19T09:30:42.552354+0000","last_became_peered":"2024-09-19T09:30:42.552354+0000","last_unstale":"2024-09-19T09:32:17.393230+0000","last_undegraded":"2024-09-19T09:32:17.393230+0000","last_fullsized":"2024-09-19T09:32:17.393230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:45.729592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398226+0000","last_change":"2024-09-19T09:30:42.561152+0000","last_active":"2024-09-19T09:32:17.398226+0000","last_peered":"2024-09-19T09:32:17.398226+0000","last_clean":"2024-09-19T09:32:17.398226+0000","last_became_active":"2024-09-19T09:30:42.560818+0000","last_became_peered":"2024-09-19T09:30:42.560818+0000","last_unstale":"2024-09-19T09:32:17.398226+0000","last_undegraded":"2024-09-19T09:32:17.398226+0000","last_fullsized":"2024-09-19T09:32:17.398226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:12.350587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398179+0000","last_change":"2024-09-19T09:30:42.561151+0000","last_active":"2024-09-19T09:32:17.398179+0000","last_peered":"2024-09-19T09:32:17.398179+0000","last_clean":"2024-09-19T09:32:17.398179+0000","last_became_active":"2024-09-19T09:30:42.560375+0000","last_became_peered":"2024-09-19T09:30:42.560375+0000","last_unstale":"2024-09-19T09:32:17.398179+0000","last_undegraded":"2024-09-19T09:32:17.398179+0000","last_fullsized":"2024-09-19T09:32:17.398179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:03.103020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.382735+0000","last_change":"2024-09-19T09:30:42.562276+0000","last_active":"2024-09-19T09:32:17.382735+0000","last_peered":"2024-09-19T09:32:17.382735+0000","last_clean":"2024-09-19T09:32:17.382735+0000","last_became_active":"2024-09-19T09:30:42.561308+0000","last_became_peered":"2024-09-19T09:30:42.561308+0000","last_unstale":"2024-09-19T09:32:17.382735+0000","last_undegraded":"2024-09-19T09:32:17.382735+0000","last_fullsized":"2024-09-19T09:32:17.382735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:39.335922+0000","last_change":"2024-09-19T09:30:40.533756+0000","last_active":"2024-09-19T09:32:39.335922+0000","last_peered":"2024-09-19T09:32:39.335922+0000","last_clean":"2024-09-19T09:32:39.335922+0000","last_became_active":"2024-09-19T09:30:40.533363+0000","last_became_peered":"2024-09-19T09:30:40.533363+0000","last_unstale":"2024-09-19T09:32:39.335922+0000","last_undegraded":"2024-09-19T09:32:39.335922+0000","last_fullsized":"2024-09-19T09:32:39.335922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:47.314178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394790+0000","last_change":"2024-09-19T09:30:42.571843+0000","last_active":"2024-09-19T09:32:17.394790+0000","last_peered":"2024-09-19T09:32:17.394790+0000","last_clean":"2024-09-19T09:32:17.394790+0000","last_became_active":"2024-09-19T09:30:42.571249+0000","last_became_peered":"2024-09-19T09:30:42.571249+0000","last_unstale":"2024-09-19T09:32:17.394790+0000","last_undegraded":"2024-09-19T09:32:17.394790+0000","last_fullsized":"2024-09-19T09:32:17.394790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:20.840950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.373206+0000","last_change":"2024-09-19T09:30:40.533783+0000","last_active":"2024-09-19T09:31:49.373206+0000","last_peered":"2024-09-19T09:31:49.373206+0000","last_clean":"2024-09-19T09:31:49.373206+0000","last_became_active":"2024-09-19T09:30:40.532582+0000","last_became_peered":"2024-09-19T09:30:40.532582+0000","last_unstale":"2024-09-19T09:31:49.373206+0000","last_undegraded":"2024-09-19T09:31:49.373206+0000","last_fullsized":"2024-09-19T09:31:49.373206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:24.293240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362498+0000","last_change":"2024-09-19T09:30:40.525924+0000","last_active":"2024-09-19T09:31:49.362498+0000","last_peered":"2024-09-19T09:31:49.362498+0000","last_clean":"2024-09-19T09:31:49.362498+0000","last_became_active":"2024-09-19T09:30:40.525590+0000","last_became_peered":"2024-09-19T09:30:40.525590+0000","last_unstale":"2024-09-19T09:31:49.362498+0000","last_undegraded":"2024-09-19T09:31:49.362498+0000","last_fullsized":"2024-09-19T09:31:49.362498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:56.644985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395177+0000","last_change":"2024-09-19T09:30:42.550661+0000","last_active":"2024-09-19T09:32:17.395177+0000","last_peered":"2024-09-19T09:32:17.395177+0000","last_clean":"2024-09-19T09:32:17.395177+0000","last_became_active":"2024-09-19T09:30:42.549775+0000","last_became_peered":"2024-09-19T09:30:42.549775+0000","last_unstale":"2024-09-19T09:32:17.395177+0000","last_undegraded":"2024-09-19T09:32:17.395177+0000","last_fullsized":"2024-09-19T09:32:17.395177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:09.150924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372719+0000","last_change":"2024-09-19T09:30:40.531805+0000","last_active":"2024-09-19T09:31:49.372719+0000","last_peered":"2024-09-19T09:31:49.372719+0000","last_clean":"2024-09-19T09:31:49.372719+0000","last_became_active":"2024-09-19T09:30:40.531469+0000","last_became_peered":"2024-09-19T09:30:40.531469+0000","last_unstale":"2024-09-19T09:31:49.372719+0000","last_undegraded":"2024-09-19T09:31:49.372719+0000","last_fullsized":"2024-09-19T09:31:49.372719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:25.712221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392420+0000","last_change":"2024-09-19T09:30:42.556312+0000","last_active":"2024-09-19T09:32:17.392420+0000","last_peered":"2024-09-19T09:32:17.392420+0000","last_clean":"2024-09-19T09:32:17.392420+0000","last_became_active":"2024-09-19T09:30:42.555969+0000","last_became_peered":"2024-09-19T09:30:42.555969+0000","last_unstale":"2024-09-19T09:32:17.392420+0000","last_undegraded":"2024-09-19T09:32:17.392420+0000","last_fullsized":"2024-09-19T09:32:17.392420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:46.310230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":238,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:38.333921+0000","last_change":"2024-09-19T09:30:40.533378+0000","last_active":"2024-09-19T09:32:38.333921+0000","last_peered":"2024-09-19T09:32:38.333921+0000","last_clean":"2024-09-19T09:32:38.333921+0000","last_became_active":"2024-09-19T09:30:40.533072+0000","last_became_peered":"2024-09-19T09:30:40.533072+0000","last_unstale":"2024-09-19T09:32:38.333921+0000","last_undegraded":"2024-09-19T09:32:38.333921+0000","last_fullsized":"2024-09-19T09:32:38.333921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:28.580385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395722+0000","last_change":"2024-09-19T09:30:42.565410+0000","last_active":"2024-09-19T09:32:17.395722+0000","last_peered":"2024-09-19T09:32:17.395722+0000","last_clean":"2024-09-19T09:32:17.395722+0000","last_became_active":"2024-09-19T09:30:42.564802+0000","last_became_peered":"2024-09-19T09:30:42.564802+0000","last_unstale":"2024-09-19T09:32:17.395722+0000","last_undegraded":"2024-09-19T09:32:17.395722+0000","last_fullsized":"2024-09-19T09:32:17.395722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:56.426484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370759+0000","last_change":"2024-09-19T09:30:40.535050+0000","last_active":"2024-09-19T09:31:49.370759+0000","last_peered":"2024-09-19T09:31:49.370759+0000","last_clean":"2024-09-19T09:31:49.370759+0000","last_became_active":"2024-09-19T09:30:40.534677+0000","last_became_peered":"2024-09-19T09:30:40.534677+0000","last_unstale":"2024-09-19T09:31:49.370759+0000","last_undegraded":"2024-09-19T09:31:49.370759+0000","last_fullsized":"2024-09-19T09:31:49.370759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:12.420661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399562+0000","last_change":"2024-09-19T09:30:42.563965+0000","last_active":"2024-09-19T09:32:17.399562+0000","last_peered":"2024-09-19T09:32:17.399562+0000","last_clean":"2024-09-19T09:32:17.399562+0000","last_became_active":"2024-09-19T09:30:42.563356+0000","last_became_peered":"2024-09-19T09:30:42.563356+0000","last_unstale":"2024-09-19T09:32:17.399562+0000","last_undegraded":"2024-09-19T09:32:17.399562+0000","last_fullsized":"2024-09-19T09:32:17.399562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:14.259229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372495+0000","last_change":"2024-09-19T09:30:40.592476+0000","last_active":"2024-09-19T09:31:49.372495+0000","last_peered":"2024-09-19T09:31:49.372495+0000","last_clean":"2024-09-19T09:31:49.372495+0000","last_became_active":"2024-09-19T09:30:40.592062+0000","last_became_peered":"2024-09-19T09:30:40.592062+0000","last_unstale":"2024-09-19T09:31:49.372495+0000","last_undegraded":"2024-09-19T09:31:49.372495+0000","last_fullsized":"2024-09-19T09:31:49.372495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:15.994447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398904+0000","last_change":"2024-09-19T09:31:49.372078+0000","last_active":"2024-09-19T09:32:17.398904+0000","last_peered":"2024-09-19T09:32:17.398904+0000","last_clean":"2024-09-19T09:32:17.398904+0000","last_became_active":"2024-09-19T09:31:49.371636+0000","last_became_peered":"2024-09-19T09:31:49.371636+0000","last_unstale":"2024-09-19T09:32:17.398904+0000","last_undegraded":"2024-09-19T09:32:17.398904+0000","last_fullsized":"2024-09-19T09:32:17.398904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:09.372482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:38.349877+0000","last_change":"2024-09-19T09:30:40.534833+0000","last_active":"2024-09-19T09:32:38.349877+0000","last_peered":"2024-09-19T09:32:38.349877+0000","last_clean":"2024-09-19T09:32:38.349877+0000","last_became_active":"2024-09-19T09:30:40.534413+0000","last_became_peered":"2024-09-19T09:30:40.534413+0000","last_unstale":"2024-09-19T09:32:38.349877+0000","last_undegraded":"2024-09-19T09:32:38.349877+0000","last_fullsized":"2024-09-19T09:32:38.349877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:27.440658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68607,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":179,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.396018+0000","last_change":"2024-09-19T09:30:42.559067+0000","last_active":"2024-09-19T09:32:17.396018+0000","last_peered":"2024-09-19T09:32:17.396018+0000","last_clean":"2024-09-19T09:32:17.396018+0000","last_became_active":"2024-09-19T09:30:42.558759+0000","last_became_peered":"2024-09-19T09:30:42.558759+0000","last_unstale":"2024-09-19T09:32:17.396018+0000","last_undegraded":"2024-09-19T09:32:17.396018+0000","last_fullsized":"2024-09-19T09:32:17.396018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:21.261690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369641+0000","last_change":"2024-09-19T09:30:40.532565+0000","last_active":"2024-09-19T09:31:49.369641+0000","last_peered":"2024-09-19T09:31:49.369641+0000","last_clean":"2024-09-19T09:31:49.369641+0000","last_became_active":"2024-09-19T09:30:40.532091+0000","last_became_peered":"2024-09-19T09:30:40.532091+0000","last_unstale":"2024-09-19T09:31:49.369641+0000","last_undegraded":"2024-09-19T09:31:49.369641+0000","last_fullsized":"2024-09-19T09:31:49.369641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.233272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392531+0000","last_change":"2024-09-19T09:30:42.558981+0000","last_active":"2024-09-19T09:32:17.392531+0000","last_peered":"2024-09-19T09:32:17.392531+0000","last_clean":"2024-09-19T09:32:17.392531+0000","last_became_active":"2024-09-19T09:30:42.558642+0000","last_became_peered":"2024-09-19T09:30:42.558642+0000","last_unstale":"2024-09-19T09:32:17.392531+0000","last_undegraded":"2024-09-19T09:32:17.392531+0000","last_fullsized":"2024-09-19T09:32:17.392531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:07.931861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372577+0000","last_change":"2024-09-19T09:30:40.531848+0000","last_active":"2024-09-19T09:31:49.372577+0000","last_peered":"2024-09-19T09:31:49.372577+0000","last_clean":"2024-09-19T09:31:49.372577+0000","last_became_active":"2024-09-19T09:30:40.531456+0000","last_became_peered":"2024-09-19T09:30:40.531456+0000","last_unstale":"2024-09-19T09:31:49.372577+0000","last_undegraded":"2024-09-19T09:31:49.372577+0000","last_fullsized":"2024-09-19T09:31:49.372577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:39.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.394396+0000","last_change":"2024-09-19T09:30:42.570769+0000","last_active":"2024-09-19T09:32:17.394396+0000","last_peered":"2024-09-19T09:32:17.394396+0000","last_clean":"2024-09-19T09:32:17.394396+0000","last_became_active":"2024-09-19T09:30:42.570275+0000","last_became_peered":"2024-09-19T09:30:42.570275+0000","last_unstale":"2024-09-19T09:32:17.394396+0000","last_undegraded":"2024-09-19T09:32:17.394396+0000","last_fullsized":"2024-09-19T09:32:17.394396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:42.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'9","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:35.326398+0000","last_change":"2024-09-19T09:30:40.533710+0000","last_active":"2024-09-19T09:32:35.326398+0000","last_peered":"2024-09-19T09:32:35.326398+0000","last_clean":"2024-09-19T09:32:35.326398+0000","last_became_active":"2024-09-19T09:30:40.533332+0000","last_became_peered":"2024-09-19T09:30:40.533332+0000","last_unstale":"2024-09-19T09:32:35.326398+0000","last_undegraded":"2024-09-19T09:32:35.326398+0000","last_fullsized":"2024-09-19T09:32:35.326398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:14.831143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.391631+0000","last_change":"2024-09-19T09:30:42.559832+0000","last_active":"2024-09-19T09:32:17.391631+0000","last_peered":"2024-09-19T09:32:17.391631+0000","last_clean":"2024-09-19T09:32:17.391631+0000","last_became_active":"2024-09-19T09:30:42.559504+0000","last_became_peered":"2024-09-19T09:30:42.559504+0000","last_unstale":"2024-09-19T09:32:17.391631+0000","last_undegraded":"2024-09-19T09:32:17.391631+0000","last_fullsized":"2024-09-19T09:32:17.391631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:24.975730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366460+0000","last_change":"2024-09-19T09:30:40.591869+0000","last_active":"2024-09-19T09:31:49.366460+0000","last_peered":"2024-09-19T09:31:49.366460+0000","last_clean":"2024-09-19T09:31:49.366460+0000","last_became_active":"2024-09-19T09:30:40.591497+0000","last_became_peered":"2024-09-19T09:30:40.591497+0000","last_unstale":"2024-09-19T09:31:49.366460+0000","last_undegraded":"2024-09-19T09:31:49.366460+0000","last_fullsized":"2024-09-19T09:31:49.366460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.439135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400654+0000","last_change":"2024-09-19T09:30:42.571129+0000","last_active":"2024-09-19T09:32:17.400654+0000","last_peered":"2024-09-19T09:32:17.400654+0000","last_clean":"2024-09-19T09:32:17.400654+0000","last_became_active":"2024-09-19T09:30:42.570199+0000","last_became_peered":"2024-09-19T09:30:42.570199+0000","last_unstale":"2024-09-19T09:32:17.400654+0000","last_undegraded":"2024-09-19T09:32:17.400654+0000","last_fullsized":"2024-09-19T09:32:17.400654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.065789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366971+0000","last_change":"2024-09-19T09:30:40.530139+0000","last_active":"2024-09-19T09:31:49.366971+0000","last_peered":"2024-09-19T09:31:49.366971+0000","last_clean":"2024-09-19T09:31:49.366971+0000","last_became_active":"2024-09-19T09:30:40.529800+0000","last_became_peered":"2024-09-19T09:30:40.529800+0000","last_unstale":"2024-09-19T09:31:49.366971+0000","last_undegraded":"2024-09-19T09:31:49.366971+0000","last_fullsized":"2024-09-19T09:31:49.366971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:31.359439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.398690+0000","last_change":"2024-09-19T09:30:42.564086+0000","last_active":"2024-09-19T09:32:17.398690+0000","last_peered":"2024-09-19T09:32:17.398690+0000","last_clean":"2024-09-19T09:32:17.398690+0000","last_became_active":"2024-09-19T09:30:42.563572+0000","last_became_peered":"2024-09-19T09:30:42.563572+0000","last_unstale":"2024-09-19T09:32:17.398690+0000","last_undegraded":"2024-09-19T09:32:17.398690+0000","last_fullsized":"2024-09-19T09:32:17.398690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.093998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.362793+0000","last_change":"2024-09-19T09:30:40.529495+0000","last_active":"2024-09-19T09:31:49.362793+0000","last_peered":"2024-09-19T09:31:49.362793+0000","last_clean":"2024-09-19T09:31:49.362793+0000","last_became_active":"2024-09-19T09:30:40.529063+0000","last_became_peered":"2024-09-19T09:30:40.529063+0000","last_unstale":"2024-09-19T09:31:49.362793+0000","last_undegraded":"2024-09-19T09:31:49.362793+0000","last_fullsized":"2024-09-19T09:31:49.362793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:45.211772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400084+0000","last_change":"2024-09-19T09:30:42.554211+0000","last_active":"2024-09-19T09:32:17.400084+0000","last_peered":"2024-09-19T09:32:17.400084+0000","last_clean":"2024-09-19T09:32:17.400084+0000","last_became_active":"2024-09-19T09:30:42.553878+0000","last_became_peered":"2024-09-19T09:30:42.553878+0000","last_unstale":"2024-09-19T09:32:17.400084+0000","last_undegraded":"2024-09-19T09:32:17.400084+0000","last_fullsized":"2024-09-19T09:32:17.400084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:34.107395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372613+0000","last_change":"2024-09-19T09:30:40.532572+0000","last_active":"2024-09-19T09:31:49.372613+0000","last_peered":"2024-09-19T09:31:49.372613+0000","last_clean":"2024-09-19T09:31:49.372613+0000","last_became_active":"2024-09-19T09:30:40.531409+0000","last_became_peered":"2024-09-19T09:30:40.531409+0000","last_unstale":"2024-09-19T09:31:49.372613+0000","last_undegraded":"2024-09-19T09:31:49.372613+0000","last_fullsized":"2024-09-19T09:31:49.372613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:49.927848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392628+0000","last_change":"2024-09-19T09:31:49.378778+0000","last_active":"2024-09-19T09:32:17.392628+0000","last_peered":"2024-09-19T09:32:17.392628+0000","last_clean":"2024-09-19T09:32:17.392628+0000","last_became_active":"2024-09-19T09:31:49.374031+0000","last_became_peered":"2024-09-19T09:31:49.374031+0000","last_unstale":"2024-09-19T09:32:17.392628+0000","last_undegraded":"2024-09-19T09:32:17.392628+0000","last_fullsized":"2024-09-19T09:32:17.392628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:04.734231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":137,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:38.350702+0000","last_change":"2024-09-19T09:30:40.681036+0000","last_active":"2024-09-19T09:32:38.350702+0000","last_peered":"2024-09-19T09:32:38.350702+0000","last_clean":"2024-09-19T09:32:38.350702+0000","last_became_active":"2024-09-19T09:30:40.680582+0000","last_became_peered":"2024-09-19T09:30:40.680582+0000","last_unstale":"2024-09-19T09:32:38.350702+0000","last_undegraded":"2024-09-19T09:32:38.350702+0000","last_fullsized":"2024-09-19T09:32:38.350702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:06.353875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401535+0000","last_change":"2024-09-19T09:30:42.562650+0000","last_active":"2024-09-19T09:32:17.401535+0000","last_peered":"2024-09-19T09:32:17.401535+0000","last_clean":"2024-09-19T09:32:17.401535+0000","last_became_active":"2024-09-19T09:30:42.562316+0000","last_became_peered":"2024-09-19T09:30:42.562316+0000","last_unstale":"2024-09-19T09:32:17.401535+0000","last_undegraded":"2024-09-19T09:32:17.401535+0000","last_fullsized":"2024-09-19T09:32:17.401535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.695117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370248+0000","last_change":"2024-09-19T09:29:31.139501+0000","last_active":"2024-09-19T09:31:49.370248+0000","last_peered":"2024-09-19T09:31:49.370248+0000","last_clean":"2024-09-19T09:31:49.370248+0000","last_became_active":"2024-09-19T09:29:31.138984+0000","last_became_peered":"2024-09-19T09:29:31.138984+0000","last_unstale":"2024-09-19T09:31:49.370248+0000","last_undegraded":"2024-09-19T09:31:49.370248+0000","last_fullsized":"2024-09-19T09:31:49.370248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:26:05.866015+0000","last_clean_scrub_stamp":"2024-09-19T09:26:05.866015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:46.079532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369273+0000","last_change":"2024-09-19T09:30:40.531061+0000","last_active":"2024-09-19T09:31:49.369273+0000","last_peered":"2024-09-19T09:31:49.369273+0000","last_clean":"2024-09-19T09:31:49.369273+0000","last_became_active":"2024-09-19T09:30:40.530680+0000","last_became_peered":"2024-09-19T09:30:40.530680+0000","last_unstale":"2024-09-19T09:31:49.369273+0000","last_undegraded":"2024-09-19T09:31:49.369273+0000","last_fullsized":"2024-09-19T09:31:49.369273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:56.900614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.392862+0000","last_change":"2024-09-19T09:30:42.562614+0000","last_active":"2024-09-19T09:32:17.392862+0000","last_peered":"2024-09-19T09:32:17.392862+0000","last_clean":"2024-09-19T09:32:17.392862+0000","last_became_active":"2024-09-19T09:30:42.561767+0000","last_became_peered":"2024-09-19T09:30:42.561767+0000","last_unstale":"2024-09-19T09:32:17.392862+0000","last_undegraded":"2024-09-19T09:32:17.392862+0000","last_fullsized":"2024-09-19T09:32:17.392862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.143554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.370098+0000","last_change":"2024-09-19T09:30:40.531416+0000","last_active":"2024-09-19T09:31:49.370098+0000","last_peered":"2024-09-19T09:31:49.370098+0000","last_clean":"2024-09-19T09:31:49.370098+0000","last_became_active":"2024-09-19T09:30:40.530238+0000","last_became_peered":"2024-09-19T09:30:40.530238+0000","last_unstale":"2024-09-19T09:31:49.370098+0000","last_undegraded":"2024-09-19T09:31:49.370098+0000","last_fullsized":"2024-09-19T09:31:49.370098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:11.708583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388022+0000","last_change":"2024-09-19T09:30:42.561195+0000","last_active":"2024-09-19T09:32:17.388022+0000","last_peered":"2024-09-19T09:32:17.388022+0000","last_clean":"2024-09-19T09:32:17.388022+0000","last_became_active":"2024-09-19T09:30:42.560885+0000","last_became_peered":"2024-09-19T09:30:42.560885+0000","last_unstale":"2024-09-19T09:32:17.388022+0000","last_undegraded":"2024-09-19T09:32:17.388022+0000","last_fullsized":"2024-09-19T09:32:17.388022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:11.563851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:36.327126+0000","last_change":"2024-09-19T09:30:40.591649+0000","last_active":"2024-09-19T09:32:36.327126+0000","last_peered":"2024-09-19T09:32:36.327126+0000","last_clean":"2024-09-19T09:32:36.327126+0000","last_became_active":"2024-09-19T09:30:40.591291+0000","last_became_peered":"2024-09-19T09:30:40.591291+0000","last_unstale":"2024-09-19T09:32:36.327126+0000","last_undegraded":"2024-09-19T09:32:36.327126+0000","last_fullsized":"2024-09-19T09:32:36.327126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:11.460452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.400420+0000","last_change":"2024-09-19T09:30:42.571004+0000","last_active":"2024-09-19T09:32:17.400420+0000","last_peered":"2024-09-19T09:32:17.400420+0000","last_clean":"2024-09-19T09:32:17.400420+0000","last_became_active":"2024-09-19T09:30:42.570253+0000","last_became_peered":"2024-09-19T09:30:42.570253+0000","last_unstale":"2024-09-19T09:32:17.400420+0000","last_undegraded":"2024-09-19T09:32:17.400420+0000","last_fullsized":"2024-09-19T09:32:17.400420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:19.406737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:35.326317+0000","last_change":"2024-09-19T09:30:40.533231+0000","last_active":"2024-09-19T09:32:35.326317+0000","last_peered":"2024-09-19T09:32:35.326317+0000","last_clean":"2024-09-19T09:32:35.326317+0000","last_became_active":"2024-09-19T09:30:40.532709+0000","last_became_peered":"2024-09-19T09:30:40.532709+0000","last_unstale":"2024-09-19T09:32:35.326317+0000","last_undegraded":"2024-09-19T09:32:35.326317+0000","last_fullsized":"2024-09-19T09:32:35.326317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:59.145049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399440+0000","last_change":"2024-09-19T09:30:42.564305+0000","last_active":"2024-09-19T09:32:17.399440+0000","last_peered":"2024-09-19T09:32:17.399440+0000","last_clean":"2024-09-19T09:32:17.399440+0000","last_became_active":"2024-09-19T09:30:42.563967+0000","last_became_peered":"2024-09-19T09:30:42.563967+0000","last_unstale":"2024-09-19T09:32:17.399440+0000","last_undegraded":"2024-09-19T09:32:17.399440+0000","last_fullsized":"2024-09-19T09:32:17.399440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:28.241435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366895+0000","last_change":"2024-09-19T09:30:40.592438+0000","last_active":"2024-09-19T09:31:49.366895+0000","last_peered":"2024-09-19T09:31:49.366895+0000","last_clean":"2024-09-19T09:31:49.366895+0000","last_became_active":"2024-09-19T09:30:40.592092+0000","last_became_peered":"2024-09-19T09:30:40.592092+0000","last_unstale":"2024-09-19T09:31:49.366895+0000","last_undegraded":"2024-09-19T09:31:49.366895+0000","last_fullsized":"2024-09-19T09:31:49.366895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.441855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388330+0000","last_change":"2024-09-19T09:30:42.557013+0000","last_active":"2024-09-19T09:32:17.388330+0000","last_peered":"2024-09-19T09:32:17.388330+0000","last_clean":"2024-09-19T09:32:17.388330+0000","last_became_active":"2024-09-19T09:30:42.556593+0000","last_became_peered":"2024-09-19T09:30:42.556593+0000","last_unstale":"2024-09-19T09:32:17.388330+0000","last_undegraded":"2024-09-19T09:32:17.388330+0000","last_fullsized":"2024-09-19T09:32:17.388330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:53.272173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.376724+0000","last_change":"2024-09-19T09:31:49.376724+0000","last_active":"2024-09-19T09:31:49.376724+0000","last_peered":"2024-09-19T09:31:49.376724+0000","last_clean":"2024-09-19T09:31:49.376724+0000","last_became_active":"2024-09-19T09:31:49.368109+0000","last_became_peered":"2024-09-19T09:31:49.368109+0000","last_unstale":"2024-09-19T09:31:49.376724+0000","last_undegraded":"2024-09-19T09:31:49.376724+0000","last_fullsized":"2024-09-19T09:31:49.376724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:42.736196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.388807+0000","last_change":"2024-09-19T09:30:42.550557+0000","last_active":"2024-09-19T09:32:17.388807+0000","last_peered":"2024-09-19T09:32:17.388807+0000","last_clean":"2024-09-19T09:32:17.388807+0000","last_became_active":"2024-09-19T09:30:42.550240+0000","last_became_peered":"2024-09-19T09:30:42.550240+0000","last_unstale":"2024-09-19T09:32:17.388807+0000","last_undegraded":"2024-09-19T09:32:17.388807+0000","last_fullsized":"2024-09-19T09:32:17.388807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:15.538746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:27.572366+0000","last_change":"2024-09-19T09:30:40.526893+0000","last_active":"2024-09-19T09:32:27.572366+0000","last_peered":"2024-09-19T09:32:27.572366+0000","last_clean":"2024-09-19T09:32:27.572366+0000","last_became_active":"2024-09-19T09:30:40.526522+0000","last_became_peered":"2024-09-19T09:30:40.526522+0000","last_unstale":"2024-09-19T09:32:27.572366+0000","last_undegraded":"2024-09-19T09:32:27.572366+0000","last_fullsized":"2024-09-19T09:32:27.572366+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:53.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.395098+0000","last_change":"2024-09-19T09:30:42.558801+0000","last_active":"2024-09-19T09:32:17.395098+0000","last_peered":"2024-09-19T09:32:17.395098+0000","last_clean":"2024-09-19T09:32:17.395098+0000","last_became_active":"2024-09-19T09:30:42.558461+0000","last_became_peered":"2024-09-19T09:30:42.558461+0000","last_unstale":"2024-09-19T09:32:17.395098+0000","last_undegraded":"2024-09-19T09:32:17.395098+0000","last_fullsized":"2024-09-19T09:32:17.395098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.371075+0000","last_change":"2024-09-19T09:30:40.681114+0000","last_active":"2024-09-19T09:31:49.371075+0000","last_peered":"2024-09-19T09:31:49.371075+0000","last_clean":"2024-09-19T09:31:49.371075+0000","last_became_active":"2024-09-19T09:30:40.680712+0000","last_became_peered":"2024-09-19T09:30:40.680712+0000","last_unstale":"2024-09-19T09:31:49.371075+0000","last_undegraded":"2024-09-19T09:31:49.371075+0000","last_fullsized":"2024-09-19T09:31:49.371075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:22.674872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399370+0000","last_change":"2024-09-19T09:30:42.564151+0000","last_active":"2024-09-19T09:32:17.399370+0000","last_peered":"2024-09-19T09:32:17.399370+0000","last_clean":"2024-09-19T09:32:17.399370+0000","last_became_active":"2024-09-19T09:30:42.563621+0000","last_became_peered":"2024-09-19T09:30:42.563621+0000","last_unstale":"2024-09-19T09:32:17.399370+0000","last_undegraded":"2024-09-19T09:32:17.399370+0000","last_fullsized":"2024-09-19T09:32:17.399370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:19.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.379065+0000","last_change":"2024-09-19T09:31:49.379065+0000","last_active":"2024-09-19T09:31:49.379065+0000","last_peered":"2024-09-19T09:31:49.379065+0000","last_clean":"2024-09-19T09:31:49.379065+0000","last_became_active":"2024-09-19T09:31:49.371819+0000","last_became_peered":"2024-09-19T09:31:49.371819+0000","last_unstale":"2024-09-19T09:31:49.379065+0000","last_undegraded":"2024-09-19T09:31:49.379065+0000","last_fullsized":"2024-09-19T09:31:49.379065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:02.748057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.359730+0000","last_change":"2024-09-19T09:30:42.557802+0000","last_active":"2024-09-19T09:32:17.359730+0000","last_peered":"2024-09-19T09:32:17.359730+0000","last_clean":"2024-09-19T09:32:17.359730+0000","last_became_active":"2024-09-19T09:30:42.557458+0000","last_became_peered":"2024-09-19T09:30:42.557458+0000","last_unstale":"2024-09-19T09:32:17.359730+0000","last_undegraded":"2024-09-19T09:32:17.359730+0000","last_fullsized":"2024-09-19T09:32:17.359730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:03:37.425774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372899+0000","last_change":"2024-09-19T09:30:40.530838+0000","last_active":"2024-09-19T09:31:49.372899+0000","last_peered":"2024-09-19T09:31:49.372899+0000","last_clean":"2024-09-19T09:31:49.372899+0000","last_became_active":"2024-09-19T09:30:40.530504+0000","last_became_peered":"2024-09-19T09:30:40.530504+0000","last_unstale":"2024-09-19T09:31:49.372899+0000","last_undegraded":"2024-09-19T09:31:49.372899+0000","last_fullsized":"2024-09-19T09:31:49.372899+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:14.601551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.399037+0000","last_change":"2024-09-19T09:30:42.561402+0000","last_active":"2024-09-19T09:32:17.399037+0000","last_peered":"2024-09-19T09:32:17.399037+0000","last_clean":"2024-09-19T09:32:17.399037+0000","last_became_active":"2024-09-19T09:30:42.560832+0000","last_became_peered":"2024-09-19T09:30:42.560832+0000","last_unstale":"2024-09-19T09:32:17.399037+0000","last_undegraded":"2024-09-19T09:32:17.399037+0000","last_fullsized":"2024-09-19T09:32:17.399037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:53.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.369583+0000","last_change":"2024-09-19T09:30:40.530822+0000","last_active":"2024-09-19T09:31:49.369583+0000","last_peered":"2024-09-19T09:31:49.369583+0000","last_clean":"2024-09-19T09:31:49.369583+0000","last_became_active":"2024-09-19T09:30:40.530436+0000","last_became_peered":"2024-09-19T09:30:40.530436+0000","last_unstale":"2024-09-19T09:31:49.369583+0000","last_undegraded":"2024-09-19T09:31:49.369583+0000","last_fullsized":"2024-09-19T09:31:49.369583+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:29.397903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.390044+0000","last_change":"2024-09-19T09:30:42.560790+0000","last_active":"2024-09-19T09:32:17.390044+0000","last_peered":"2024-09-19T09:32:17.390044+0000","last_clean":"2024-09-19T09:32:17.390044+0000","last_became_active":"2024-09-19T09:30:42.560494+0000","last_became_peered":"2024-09-19T09:30:42.560494+0000","last_unstale":"2024-09-19T09:32:17.390044+0000","last_undegraded":"2024-09-19T09:32:17.390044+0000","last_fullsized":"2024-09-19T09:32:17.390044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:23.243320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.377852+0000","last_change":"2024-09-19T09:31:49.377852+0000","last_active":"2024-09-19T09:31:49.377852+0000","last_peered":"2024-09-19T09:31:49.377852+0000","last_clean":"2024-09-19T09:31:49.377852+0000","last_became_active":"2024-09-19T09:31:49.372621+0000","last_became_peered":"2024-09-19T09:31:49.372621+0000","last_unstale":"2024-09-19T09:31:49.377852+0000","last_undegraded":"2024-09-19T09:31:49.377852+0000","last_fullsized":"2024-09-19T09:31:49.377852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:11.747447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401138+0000","last_change":"2024-09-19T09:30:42.562252+0000","last_active":"2024-09-19T09:32:17.401138+0000","last_peered":"2024-09-19T09:32:17.401138+0000","last_clean":"2024-09-19T09:32:17.401138+0000","last_became_active":"2024-09-19T09:30:42.561933+0000","last_became_peered":"2024-09-19T09:30:42.561933+0000","last_unstale":"2024-09-19T09:32:17.401138+0000","last_undegraded":"2024-09-19T09:32:17.401138+0000","last_fullsized":"2024-09-19T09:32:17.401138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:01.807679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401513+0000","last_change":"2024-09-19T09:30:42.571386+0000","last_active":"2024-09-19T09:32:17.401513+0000","last_peered":"2024-09-19T09:32:17.401513+0000","last_clean":"2024-09-19T09:32:17.401513+0000","last_became_active":"2024-09-19T09:30:42.570990+0000","last_became_peered":"2024-09-19T09:30:42.570990+0000","last_unstale":"2024-09-19T09:32:17.401513+0000","last_undegraded":"2024-09-19T09:32:17.401513+0000","last_fullsized":"2024-09-19T09:32:17.401513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:42.722827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366976+0000","last_change":"2024-09-19T09:30:40.530454+0000","last_active":"2024-09-19T09:31:49.366976+0000","last_peered":"2024-09-19T09:31:49.366976+0000","last_clean":"2024-09-19T09:31:49.366976+0000","last_became_active":"2024-09-19T09:30:40.529846+0000","last_became_peered":"2024-09-19T09:30:40.529846+0000","last_unstale":"2024-09-19T09:31:49.366976+0000","last_undegraded":"2024-09-19T09:31:49.366976+0000","last_fullsized":"2024-09-19T09:31:49.366976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.091632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.397204+0000","last_change":"2024-09-19T09:30:42.564983+0000","last_active":"2024-09-19T09:32:17.397204+0000","last_peered":"2024-09-19T09:32:17.397204+0000","last_clean":"2024-09-19T09:32:17.397204+0000","last_became_active":"2024-09-19T09:30:42.564633+0000","last_became_peered":"2024-09-19T09:30:42.564633+0000","last_unstale":"2024-09-19T09:32:17.397204+0000","last_undegraded":"2024-09-19T09:32:17.397204+0000","last_fullsized":"2024-09-19T09:32:17.397204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:59.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.366959+0000","last_change":"2024-09-19T09:30:40.536480+0000","last_active":"2024-09-19T09:31:49.366959+0000","last_peered":"2024-09-19T09:31:49.366959+0000","last_clean":"2024-09-19T09:31:49.366959+0000","last_became_active":"2024-09-19T09:30:40.536125+0000","last_became_peered":"2024-09-19T09:30:40.536125+0000","last_unstale":"2024-09-19T09:31:49.366959+0000","last_undegraded":"2024-09-19T09:31:49.366959+0000","last_fullsized":"2024-09-19T09:31:49.366959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:27.848679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.401770+0000","last_change":"2024-09-19T09:30:42.558914+0000","last_active":"2024-09-19T09:32:17.401770+0000","last_peered":"2024-09-19T09:32:17.401770+0000","last_clean":"2024-09-19T09:32:17.401770+0000","last_became_active":"2024-09-19T09:30:42.558620+0000","last_became_peered":"2024-09-19T09:30:42.558620+0000","last_unstale":"2024-09-19T09:32:17.401770+0000","last_undegraded":"2024-09-19T09:32:17.401770+0000","last_fullsized":"2024-09-19T09:32:17.401770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:21.541124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:31:49.372452+0000","last_change":"2024-09-19T09:30:40.591961+0000","last_active":"2024-09-19T09:31:49.372452+0000","last_peered":"2024-09-19T09:31:49.372452+0000","last_clean":"2024-09-19T09:31:49.372452+0000","last_became_active":"2024-09-19T09:30:40.591522+0000","last_became_peered":"2024-09-19T09:30:40.591522+0000","last_unstale":"2024-09-19T09:31:49.372452+0000","last_undegraded":"2024-09-19T09:31:49.372452+0000","last_fullsized":"2024-09-19T09:31:49.372452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.750487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:17.393001+0000","last_change":"2024-09-19T09:30:42.561603+0000","last_active":"2024-09-19T09:32:17.393001+0000","last_peered":"2024-09-19T09:32:17.393001+0000","last_clean":"2024-09-19T09:32:17.393001+0000","last_became_active":"2024-09-19T09:30:42.561290+0000","last_became_peered":"2024-09-19T09:30:42.561290+0000","last_unstale":"2024-09-19T09:32:17.393001+0000","last_undegraded":"2024-09-19T09:32:17.393001+0000","last_fullsized":"2024-09-19T09:32:17.393001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:41.526039+0000","last_clean_scrub_stamp":"2024-09-19T09:30:41.526039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:23.097142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T09:32:36.999102+0000","last_change":"2024-09-19T09:30:40.534311+0000","last_active":"2024-09-19T09:32:36.999102+0000","last_peered":"2024-09-19T09:32:36.999102+0000","last_clean":"2024-09-19T09:32:36.999102+0000","last_became_active":"2024-09-19T09:30:40.533939+0000","last_became_peered":"2024-09-19T09:30:40.533939+0000","last_unstale":"2024-09-19T09:32:36.999102+0000","last_undegraded":"2024-09-19T09:32:36.999102+0000","last_fullsized":"2024-09-19T09:32:36.999102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:30:39.514966+0000","last_clean_scrub_stamp":"2024-09-19T09:30:39.514966+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:53.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77725,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1100,"num_read_kb":636,"num_write":267,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1601536,"data_stored":1093053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":432229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":432378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1026920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1024829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":502496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":501883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":433561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:32:42.731 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:43.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:42 smithi028 ceph-mon[30972]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:32:43.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:42 smithi028 ceph-mon[30972]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:42 smithi078 ceph-mon[35619]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:32:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:42 smithi078 ceph-mon[35619]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:43.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:42 smithi038 ceph-mon[35271]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:32:43.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:42 smithi038 ceph-mon[35271]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:32:43.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 -- 172.21.15.28:0/2026362009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 msgr2=0x7f38a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 --2- 172.21.15.28:0/2026362009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a81067e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3894009900 tx=0x7f389402a410 comp rx=0 tx=0).stop 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 -- 172.21.15.28:0/2026362009 shutdown_connections 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 --2- 172.21.15.28:0/2026362009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8106f10 0x7f38a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 --2- 172.21.15.28:0/2026362009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a81067e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 --2- 172.21.15.28:0/2026362009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101de0 0x7f38a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.497+0000 7f38af122640 1 -- 172.21.15.28:0/2026362009 >> 172.21.15.28:0/2026362009 conn(0x7f38a80fd8c0 msgr2=0x7f38a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- 172.21.15.28:0/2026362009 shutdown_connections 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- 172.21.15.28:0/2026362009 wait complete. 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 Processor -- start 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- start start 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8101de0 0x7f38a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:43.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 0x7f38a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8111f30 con 0x7f38a8102790 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8111db0 con 0x7f38a8101de0 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38af122640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81120b0 con 0x7f38a8106f10 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.498+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54160/0 (socket says 172.21.15.28:54160) 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 -- 172.21.15.28:0/1001519184 learned_addr learned my addr 172.21.15.28:0/1001519184 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38acebe640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8101de0 0x7f38a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38ad6bf640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 0x7f38a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 -- 172.21.15.28:0/1001519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 msgr2=0x7f38a819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 0x7f38a819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 -- 172.21.15.28:0/1001519184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8101de0 msgr2=0x7f38a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:43.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8101de0 0x7f38a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 -- 172.21.15.28:0/1001519184 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819f620 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38ad6bf640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 0x7f38a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a7fff640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3894006010 tx=0x7f38940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389403f030 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3894003ce0 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894035070 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819f8b0 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.499+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81a71f0 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.500+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a81032c0 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.502+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f3894042030 con 0x7f38a8102790 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.503+0000 7f38a5ffb640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 0x7f38980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:43.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.503+0000 7f38acebe640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 0x7f38980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:43.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.503+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f38940f0430 con 0x7f38a8102790 2024-09-19T09:32:43.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.503+0000 7f38acebe640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 0x7f38980a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f389c005d00 tx=0x7f389c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:43.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.506+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38940bd080 con 0x7f38a8102790 2024-09-19T09:32:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.732+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a81021c0 con 0x7f38a8102790 2024-09-19T09:32:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.733+0000 7f38a5ffb640 1 -- 172.21.15.28:0/1001519184 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f38a81021c0 con 0x7f38a8102790 2024-09-19T09:32:43.735 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:43.736 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:32:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:32:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 msgr2=0x7f38980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 0x7f38980a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f389c005d00 tx=0x7f389c00b410 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 msgr2=0x7f38a819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3894006010 tx=0x7f38940042f0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 shutdown_connections 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f38980a32d0 0x7f38980a5790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8106f10 0x7f38a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38a8102790 0x7f38a819abb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 --2- 172.21.15.28:0/1001519184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38a8101de0 0x7f38a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.735+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 >> 172.21.15.28:0/1001519184 conn(0x7f38a80fd8c0 msgr2=0x7f38a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.736+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 shutdown_connections 2024-09-19T09:32:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:43.736+0000 7f38af122640 1 -- 172.21.15.28:0/1001519184 wait complete. 2024-09-19T09:32:43.826 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:43 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:32:44.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:43 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:32:44.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:43 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1001519184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:44 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1001519184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:44 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.120 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:45.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:32:45.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1001519184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:45.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:44 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:45.405 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- 172.21.15.28:0/774000861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 msgr2=0x7f74f0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/774000861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f0101200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec02a480 comp rx=0 tx=0).stop 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- 172.21.15.28:0/774000861 shutdown_connections 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/774000861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 0x7f74f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/774000861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 0x7f74f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/774000861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f0101200 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- 172.21.15.28:0/774000861 >> 172.21.15.28:0/774000861 conn(0x7f74f00fc7d0 msgr2=0x7f74f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:45.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- 172.21.15.28:0/774000861 shutdown_connections 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- 172.21.15.28:0/774000861 wait complete. 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 Processor -- start 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.837+0000 7f74f7ba3640 1 -- start start 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 0x7f74f01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 0x7f74f0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0110fa0 con 0x7f74f0100e20 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0110e20 con 0x7f74f0102570 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f7ba3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0111120 con 0x7f74f0102ef0 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54168/0 (socket says 172.21.15.28:54168) 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 -- 172.21.15.28:0/2479678394 learned_addr learned my addr 172.21.15.28:0/2479678394 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f513e640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 0x7f74f01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f6140640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 0x7f74f0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 -- 172.21.15.28:0/2479678394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 msgr2=0x7f74f0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 0x7f74f0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 -- 172.21.15.28:0/2479678394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 msgr2=0x7f74f01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 0x7f74f01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 -- 172.21.15.28:0/2479678394 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f01a6a10 con 0x7f74f0100e20 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.838+0000 7f74f593f640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006010 tx=0x7f74ec003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.839+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec03f070 con 0x7f74f0100e20 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.839+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01a6ca0 con 0x7f74f0100e20 2024-09-19T09:32:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.839+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec031e30 con 0x7f74f0100e20 2024-09-19T09:32:45.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.839+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a7200 con 0x7f74f0100e20 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.839+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec035030 con 0x7f74f0100e20 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.840+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f0101200 con 0x7f74f0100e20 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.842+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f74ec042070 con 0x7f74f0100e20 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.843+0000 7f74e2ffd640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 0x7f74dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.843+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f74ec0eff70 con 0x7f74f0100e20 2024-09-19T09:32:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.843+0000 7f74f513e640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 0x7f74dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:45.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.844+0000 7f74f513e640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 0x7f74dc0a5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74f0199300 tx=0x7f74e400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:45.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:45.847+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f74ec0bcc90 con 0x7f74f0100e20 2024-09-19T09:32:46.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.064+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74f0109090 con 0x7f74f0100e20 2024-09-19T09:32:46.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.064+0000 7f74e2ffd640 1 -- 172.21.15.28:0/2479678394 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f74ec0bc590 con 0x7f74f0100e20 2024-09-19T09:32:46.066 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:46.067 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:32:46.067 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:32:46.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 msgr2=0x7f74dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:46.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 0x7f74dc0a5810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74f0199300 tx=0x7f74e400a4c0 comp rx=0 tx=0).stop 2024-09-19T09:32:46.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 msgr2=0x7f74f01971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:46.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006010 tx=0x7f74ec003c80 comp rx=0 tx=0).stop 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 shutdown_connections 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f74dc0a3350 0x7f74dc0a5810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74f0102570 0x7f74f01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74f0102ef0 0x7f74f0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 --2- 172.21.15.28:0/2479678394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74f0100e20 0x7f74f01971a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 >> 172.21.15.28:0/2479678394 conn(0x7f74f00fc7d0 msgr2=0x7f74f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 shutdown_connections 2024-09-19T09:32:46.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:46.067+0000 7f74f7ba3640 1 -- 172.21.15.28:0/2479678394 wait complete. 2024-09-19T09:32:46.717 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.718 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:46 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2479678394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:46.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:46 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2479678394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:32:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:46 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2479678394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:47.003 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.452+0000 7f4af8c63640 1 -- 172.21.15.28:0/2138351912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 msgr2=0x7f4af410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.452+0000 7f4af8c63640 1 --2- 172.21.15.28:0/2138351912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af410e610 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009ca0 tx=0x7f4aec02a950 comp rx=0 tx=0).stop 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 -- 172.21.15.28:0/2138351912 shutdown_connections 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 --2- 172.21.15.28:0/2138351912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af410e610 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 --2- 172.21.15.28:0/2138351912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4103660 0x7f4af410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 --2- 172.21.15.28:0/2138351912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4102d40 0x7f4af4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 -- 172.21.15.28:0/2138351912 >> 172.21.15.28:0/2138351912 conn(0x7f4af40fc7a0 msgr2=0x7f4af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 -- 172.21.15.28:0/2138351912 shutdown_connections 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 -- 172.21.15.28:0/2138351912 wait complete. 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 Processor -- start 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.453+0000 7f4af8c63640 1 -- start start 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 0x7f4af4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 0x7f4af4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4111f30 con 0x7f4af410c170 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4111db0 con 0x7f4af4103660 2024-09-19T09:32:47.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af8c63640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41120b0 con 0x7f4af4102d40 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 0x7f4af4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 0x7f4af4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af2d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54186/0 (socket says 172.21.15.28:54186) 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af2d9d640 1 -- 172.21.15.28:0/4079131040 learned_addr learned my addr 172.21.15.28:0/4079131040 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 0x7f4af4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:51922/0 (socket says 172.21.15.28:51922) 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 -- 172.21.15.28:0/4079131040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 msgr2=0x7f4af4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 0x7f4af4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 -- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 msgr2=0x7f4af419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.454+0000 7f4af1d9b640 1 -- 172.21.15.28:0/4079131040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af419d3f0 con 0x7f4af4103660 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.455+0000 7f4af1d9b640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 0x7f4af4198980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00d680 tx=0x7f4adc00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.462+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc017070 con 0x7f4af4103660 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.462+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4adc0097c0 con 0x7f4af4103660 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.462+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af419d6e0 con 0x7f4af4103660 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.462+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc004c60 con 0x7f4af4103660 2024-09-19T09:32:47.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.462+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41008d0 con 0x7f4af4103660 2024-09-19T09:32:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.463+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af4103120 con 0x7f4af4103660 2024-09-19T09:32:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.464+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f4adc009970 con 0x7f4af4103660 2024-09-19T09:32:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.465+0000 7f4aeb7fe640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 0x7f4ae00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.465+0000 7f4af259c640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 0x7f4ae00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.465+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4adc0cf230 con 0x7f4af4103660 2024-09-19T09:32:47.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.465+0000 7f4af259c640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 0x7f4ae00a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4005d00 tx=0x7f4ae400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:47.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.468+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4adc09b850 con 0x7f4af4103660 2024-09-19T09:32:47.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.645+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4af41071e0 con 0x7f4af4103660 2024-09-19T09:32:47.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.646+0000 7f4aeb7fe640 1 -- 172.21.15.28:0/4079131040 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22580 (secure 0 0 0) 0x7f4adc09fb50 con 0x7f4af4103660 2024-09-19T09:32:47.649 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:47.649 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":82,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:31:49.348824+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 msgr2=0x7f4ae00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 0x7f4ae00a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4005d00 tx=0x7f4ae400b410 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 msgr2=0x7f4af4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 0x7f4af4198980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00d680 tx=0x7f4adc00db40 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 shutdown_connections 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ae00a3350 0x7f4ae00a5810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af4103660 0x7f4af4198980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4af410c170 0x7f4af419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 --2- 172.21.15.28:0/4079131040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4af4102d40 0x7f4af4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:47.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 >> 172.21.15.28:0/4079131040 conn(0x7f4af40fc7a0 msgr2=0x7f4af40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:47.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 shutdown_connections 2024-09-19T09:32:47.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:47.649+0000 7f4af8c63640 1 -- 172.21.15.28:0/4079131040 wait complete. 2024-09-19T09:32:48.273 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T09:32:48.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:48 smithi028 ceph-mon[30972]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T09:32:48.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4079131040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:48 smithi078 ceph-mon[35619]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T09:32:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4079131040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:48.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:48 smithi038 ceph-mon[35271]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-19T09:32:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4079131040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:32:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 -- 172.21.15.28:0/1492688083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 msgr2=0x7fbd58109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 --2- 172.21.15.28:0/1492688083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd58109fb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009ca0 tx=0x7fbd5002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 -- 172.21.15.28:0/1492688083 shutdown_connections 2024-09-19T09:32:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 --2- 172.21.15.28:0/1492688083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58102d60 0x7fbd581031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 --2- 172.21.15.28:0/1492688083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd58109fb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 --2- 172.21.15.28:0/1492688083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58101b70 0x7fbd58101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 -- 172.21.15.28:0/1492688083 >> 172.21.15.28:0/1492688083 conn(0x7fbd580fd2f0 msgr2=0x7fbd580ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 -- 172.21.15.28:0/1492688083 shutdown_connections 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.183+0000 7fbd5d253640 1 -- 172.21.15.28:0/1492688083 wait complete. 2024-09-19T09:32:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 Processor -- start 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 -- start start 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58101b70 0x7fbd58197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58102d60 0x7fbd581982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5810d840 con 0x7fbd58103720 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5810d6c0 con 0x7fbd58101b70 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5d253640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5810d9c0 con 0x7fbd58102d60 2024-09-19T09:32:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54196/0 (socket says 172.21.15.28:54196) 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5759e640 1 -- 172.21.15.28:0/721228690 learned_addr learned my addr 172.21.15.28:0/721228690 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58101b70 0x7fbd58197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.184+0000 7fbd5659c640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58102d60 0x7fbd581982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 -- 172.21.15.28:0/721228690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58102d60 msgr2=0x7fbd581982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58102d60 0x7fbd581982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 -- 172.21.15.28:0/721228690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58101b70 msgr2=0x7fbd58197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58101b70 0x7fbd58197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 -- 172.21.15.28:0/721228690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5819f9d0 con 0x7fbd58103720 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5759e640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbd500319f0 tx=0x7fbd5002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd3ffff640 1 -- 172.21.15.28:0/721228690 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003c70 con 0x7fbd58103720 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd3ffff640 1 -- 172.21.15.28:0/721228690 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd50003e10 con 0x7fbd58103720 2024-09-19T09:32:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5819fc60 con 0x7fbd58103720 2024-09-19T09:32:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd3ffff640 1 -- 172.21.15.28:0/721228690 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5002fd30 con 0x7fbd58103720 2024-09-19T09:32:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.185+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd581a01b0 con 0x7fbd58103720 2024-09-19T09:32:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.187+0000 7fbd3ffff640 1 -- 172.21.15.28:0/721228690 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fbd50004040 con 0x7fbd58103720 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.187+0000 7fbd5ffeb740 1 Processor -- start 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.187+0000 7fbd5ffeb740 1 -- start start 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6420da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6423830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561ba637f2f0 con 0x561ba641b9a0 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561ba6327270 con 0x561ba641c2e0 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ba637b2b0 con 0x561ba6421370 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54212/0 (socket says 172.21.15.28:54212) 2024-09-19T09:32:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 -- 172.21.15.28:0/1869612110 learned_addr learned my addr 172.21.15.28:0/1869612110 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6420da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 -- 172.21.15.28:0/1869612110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 msgr2=0x561ba6423830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6423830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 -- 172.21.15.28:0/1869612110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 msgr2=0x561ba6420da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6420da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 -- 172.21.15.28:0/1869612110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba6326340 con 0x561ba641b9a0 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd3ffff640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 0x7fbd400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd5659c640 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00ccd0 tx=0x7fbd4c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9cd37c6f9096c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 0x7fbd400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd3ffff640 1 -- 172.21.15.28:0/721228690 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbd500f0f00 con 0x7fbd58103720 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd3d7fa640 1 -- 172.21.15.28:0/1869612110 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c016070 con 0x561ba641b9a0 2024-09-19T09:32:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.188+0000 7fbd3d7fa640 1 -- 172.21.15.28:0/1869612110 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4c012b60 con 0x561ba641b9a0 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd3d7fa640 1 -- 172.21.15.28:0/1869612110 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c011c40 con 0x561ba641b9a0 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/1869612110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 msgr2=0x561ba641bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00ccd0 tx=0x7fbd4c010740 comp rx=0 tx=0).stop 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 0x7fbd400a5790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x561ba641cd10 tx=0x7fbd4800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/1869612110 shutdown_connections 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6420da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6423830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/1869612110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba641bda0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/1869612110 >> 172.21.15.28:0/1869612110 conn(0x561ba6417010 msgr2=0x561ba6417410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/1869612110 shutdown_connections 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.189+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/1869612110 wait complete. 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 -- start start 2024-09-19T09:32:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 Processor -- start 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 0x561ba64290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6425cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561ba6424340 con 0x561ba641c2e0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561ba6308290 con 0x561ba641b9a0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ba64244c0 con 0x561ba6421370 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5ffeb740 10 client.0 _ll_register_callbacks cb 0x561ba6339140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54214/0 (socket says 172.21.15.28:54214) 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 -- 172.21.15.28:0/2673778764 learned_addr learned my addr 172.21.15.28:0/2673778764 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 0x561ba64290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5759e640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6425cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 -- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 msgr2=0x561ba6425cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6425cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 -- 172.21.15.28:0/2673778764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 msgr2=0x561ba64290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 0x561ba64290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 -- 172.21.15.28:0/2673778764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba6092210 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 0x561ba64290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00cca0 tx=0x7fbd4c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d7e4355e31bf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.190+0000 7fbd3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c016070 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd4c012b60 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbd4c004020 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd4c004220 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 10 client.15384 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ba615e620 con 0x561ba641c2e0 2024-09-19T09:32:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4c012b60 con 0x561ba641c2e0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd4c005340 con 0x561ba641c2e0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 20 client.15384 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba6439db0 0x561ba643c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 4 client.15384 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.191+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ba638ca20 con 0x561ba6439db0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.192+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba6439db0 0x561ba643c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.192+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba6439db0 0x561ba643c1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72851cc83836a92c server_cookie=6e89715cc8bf1583 in_seq=0 out_seq=0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.192+0000 7fbd3aff5640 10 client.15384 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2673778764 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ba638ca20 con 0x561ba6439db0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd3aff5640 10 client.15384 handle_command_reply: tid=0 2024-09-19T09:32:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 client.15384 shutdown 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 msgr2=0x561ba6429610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00cca0 tx=0x7fbd4c011bf0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 shutdown_connections 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba6439db0 0x561ba643c1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6425cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641c2e0 0x561ba6429610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2673778764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641b9a0 0x561ba64290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd3aff5640 0 client.15384 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 >> 172.21.15.28:0/2673778764 conn(0x561ba6417010 msgr2=0x561ba6418820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 shutdown_connections 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2673778764 wait complete. 2024-09-19T09:32:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.193+0000 7fbd5ffeb740 20 client.15384 trim_cache size 0 max 16384 2024-09-19T09:32:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 Processor -- start 2024-09-19T09:32:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 -- start start 2024-09-19T09:32:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6417ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6418400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a6c80 con 0x561ba641b9a0 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a87c0 con 0x561ba641c2e0 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a7350 con 0x561ba6421370 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.207+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54228/0 (socket says 172.21.15.28:54228) 2024-09-19T09:32:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 -- 172.21.15.28:0/4116352020 learned_addr learned my addr 172.21.15.28:0/4116352020 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd5659c640 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6417ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd5759e640 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6418400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 -- 172.21.15.28:0/4116352020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 msgr2=0x561ba6418400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6418400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 -- 172.21.15.28:0/4116352020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 msgr2=0x561ba6417ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6417ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 -- 172.21.15.28:0/4116352020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba643c510 con 0x561ba641b9a0 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x561ba6417e90 tx=0x7fbd48015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6b0ed6bd8b6590 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd3cff9640 1 -- 172.21.15.28:0/4116352020 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48005220 con 0x561ba641b9a0 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd3cff9640 1 -- 172.21.15.28:0/4116352020 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48004c80 con 0x561ba641b9a0 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.208+0000 7fbd3cff9640 1 -- 172.21.15.28:0/4116352020 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48029860 con 0x561ba641b9a0 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/4116352020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 msgr2=0x561ba6424fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x561ba6417e90 tx=0x7fbd48015000 comp rx=0 tx=0).stop 2024-09-19T09:32:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/4116352020 shutdown_connections 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba6417ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba6418400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/4116352020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba6424fd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/4116352020 >> 172.21.15.28:0/4116352020 conn(0x561ba6417010 msgr2=0x561ba64248c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/4116352020 shutdown_connections 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/4116352020 wait complete. 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- start start 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 Processor -- start 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba644ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba644ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a80f0 con 0x561ba641b9a0 2024-09-19T09:32:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a7a20 con 0x561ba641c2e0 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581a4c60 con 0x561ba6421370 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5ffeb740 10 client.0 _ll_register_callbacks cb 0x7fbd4c00cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54238/0 (socket says 172.21.15.28:54238) 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd56d9d640 1 -- 172.21.15.28:0/2733772447 learned_addr learned my addr 172.21.15.28:0/2733772447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.209+0000 7fbd5759e640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba644ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba644ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 -- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 msgr2=0x561ba644ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba644ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 -- 172.21.15.28:0/2733772447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 msgr2=0x561ba644ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba644ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 -- 172.21.15.28:0/2733772447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba644d070 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd5759e640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba644ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd56d9d640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800bae0 tx=0x7fbd48004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebea221d014a1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48005200 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbd480053a0 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd48004c80 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd480055a0 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd5ffeb740 10 client.15390 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ba644e3f0 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48004c80 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.210+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd48031970 con 0x561ba641b9a0 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:32:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 20 client.15390 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba64a5020 0x561ba64a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 4 client.15390 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ba6386d70 con 0x561ba64a5020 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba64a5020 0x561ba64a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd5659c640 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba64a5020 0x561ba64a7440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f98321923a300f0 server_cookie=6e024a4cf8b09a58 in_seq=0 out_seq=0 2024-09-19T09:32:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.211+0000 7fbd3aff5640 10 client.15390 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd3aff5640 1 -- 172.21.15.28:0/2733772447 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ba6386d70 con 0x561ba64a5020 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd3aff5640 10 client.15390 handle_command_reply: tid=0 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 client.15390 shutdown 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 msgr2=0x561ba644b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800bae0 tx=0x7fbd48004740 comp rx=0 tx=0).stop 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 shutdown_connections 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ba641c2e0 0x561ba644ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x561ba64a5020 0x561ba64a7440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ba6421370 0x561ba644ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 --2- 172.21.15.28:0/2733772447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561ba641b9a0 0x561ba644b510 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd3aff5640 0 client.15390 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 >> 172.21.15.28:0/2733772447 conn(0x561ba6417010 msgr2=0x561ba64250e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 shutdown_connections 2024-09-19T09:32:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 1 -- 172.21.15.28:0/2733772447 wait complete. 2024-09-19T09:32:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.224+0000 7fbd5ffeb740 20 client.15390 trim_cache size 0 max 16384 2024-09-19T09:32:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:32:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:32:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:32:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 msgr2=0x7fbd400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 0x7fbd400a5790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x561ba641cd10 tx=0x7fbd4800b390 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 msgr2=0x7fbd5819f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbd500319f0 tx=0x7fbd5002f450 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 shutdown_connections 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbd400a32d0 0x7fbd400a5790 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd58103720 0x7fbd5819f360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd58102d60 0x7fbd581982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 --2- 172.21.15.28:0/721228690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd58101b70 0x7fbd58197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 >> 172.21.15.28:0/721228690 conn(0x7fbd580fd2f0 msgr2=0x7fbd58106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 shutdown_connections 2024-09-19T09:32:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:32:49.225+0000 7fbd5d253640 1 -- 172.21.15.28:0/721228690 wait complete. 2024-09-19T09:32:49.558 INFO:teuthology.orchestra.run.smithi028.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T09:32:49.559 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T09:32:49.559 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T09:32:49.559 DEBUG:tasks.quiescer:Yielding 2024-09-19T09:32:49.559 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T09:32:49.574 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:32:49.574 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T09:32:49.574 INFO:tasks.workunit:timeout=3h 2024-09-19T09:32:49.574 INFO:tasks.workunit:cleanup=True 2024-09-19T09:32:49.575 DEBUG:teuthology.orchestra.run.smithi028:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:49.575 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 09:31:00.458938123 +0000 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 09:31:00.458938123 +0000 2024-09-19T09:32:49.592 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 09:32:27.017393055 +0000 2024-09-19T09:32:49.593 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 09:31:00.458938123 +0000 2024-09-19T09:32:49.593 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T09:32:49.593 DEBUG:teuthology.orchestra.run.smithi028:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T09:32:49.668 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:32:49.725 INFO:tasks.workunit.client.0.smithi028.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T09:32:49.832 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1057523302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8100e20 msgr2=0x7fd5f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1057523302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8100e20 0x7fd5f8101200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009900 tx=0x7fd5e802a480 comp rx=0 tx=0).stop 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1057523302 shutdown_connections 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1057523302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1057523302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8102ef0 0x7fd5f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1057523302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8100e20 0x7fd5f8101200 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1057523302 >> 172.21.15.28:0/1057523302 conn(0x7fd5f80fc7d0 msgr2=0x7fd5f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:50.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1057523302 shutdown_connections 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.310+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1057523302 wait complete. 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 Processor -- start 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 -- start start 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8100e20 0x7fd5f8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 0x7fd5f819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8110ff0 con 0x7fd5f8102ef0 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8110e70 con 0x7fd5f8102570 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.311+0000 7fd5fde6e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8111170 con 0x7fd5f8100e20 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 0x7fd5f819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:50.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:52582/0 (socket says 172.21.15.28:52582) 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 -- 172.21.15.28:0/1102939536 learned_addr learned my addr 172.21.15.28:0/1102939536 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f77fe640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8100e20 0x7fd5f8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 -- 172.21.15.28:0/1102939536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8100e20 msgr2=0x7fd5f8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8100e20 0x7fd5f8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 -- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 msgr2=0x7fd5f819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 0x7fd5f819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 -- 172.21.15.28:0/1102939536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f819d540 con 0x7fd5f8102570 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f7fff640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 0x7fd5f819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f6ffd640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e9f0 tx=0x7fd5ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00cd80 con 0x7fd5f8102570 2024-09-19T09:32:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f819d830 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec010910 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.312+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0188f0 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.313+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81a5110 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.313+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f8101200 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.315+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd5ec010ac0 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.316+0000 7fd5f4ff9640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 0x7fd5e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.316+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd5ec0d0260 con 0x7fd5f8102570 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.316+0000 7fd5f77fe640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 0x7fd5e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.316+0000 7fd5f77fe640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 0x7fd5e00a5790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000c00 tx=0x7fd5e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.319+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5ec09c7a0 con 0x7fd5f8102570 2024-09-19T09:32:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:50 smithi028 ceph-mon[30972]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-19T09:32:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:50 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:32:50.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.550+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5f810a0d0 con 0x7fd5f8102570 2024-09-19T09:32:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.551+0000 7fd5f4ff9640 1 -- 172.21.15.28:0/1102939536 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7fd5ec0a0aa0 con 0x7fd5f8102570 2024-09-19T09:32:50.553 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:32:50.553 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:32:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 msgr2=0x7fd5e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 0x7fd5e00a5790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000c00 tx=0x7fd5e8005b50 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 msgr2=0x7fd5f8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e9f0 tx=0x7fd5ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 shutdown_connections 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd5e00a32d0 0x7fd5e00a5790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5f8102570 0x7fd5f8198ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5f8102ef0 0x7fd5f819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 --2- 172.21.15.28:0/1102939536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f8100e20 0x7fd5f8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 >> 172.21.15.28:0/1102939536 conn(0x7fd5f80fc7d0 msgr2=0x7fd5f806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 shutdown_connections 2024-09-19T09:32:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:50.553+0000 7fd5fde6e640 1 -- 172.21.15.28:0/1102939536 wait complete. 2024-09-19T09:32:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:50 smithi078 ceph-mon[35619]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-19T09:32:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:50 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:32:50.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:50 smithi038 ceph-mon[35271]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-19T09:32:50.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:50 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:32:51.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:30:43.054690+0000', 'modified': '2024-09-19T09:32:27.697965+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24499, 'mds_1': 24482, 'mds_2': 24478, 'mds_3': 24472, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6839/1981224443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6838', 'nonce': 1981224443}, {'type': 'v1', 'addr': '172.21.15.28:6839', 'nonce': 1981224443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.28:6841/438294996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6840', 'nonce': 438294996}, {'type': 'v1', 'addr': '172.21.15.28:6841', 'nonce': 438294996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3013823435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3013823435}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3013823435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/347872214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 347872214}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 347872214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6833/2992347158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 2992347158}, {'type': 'v1', 'addr': '172.21.15.78:6833', 'nonce': 2992347158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.38:6837/560457008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 560457008}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 560457008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.78:6835/2086564778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 2086564778}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 2086564778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.78:6837/2648819818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2648819818}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2648819818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.38:6839/3114720816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 3114720816}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 3114720816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.78:6839/3581897338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3581897338}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3581897338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24455, 24472, 24478, 24482, 24499]} 2024-09-19T09:32:51.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:32:51.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:32:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:51.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:51 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1102939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:51 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1102939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:51.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:51 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1102939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 -- 172.21.15.28:0/3735685427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8103660 msgr2=0x7f19b810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 --2- 172.21.15.28:0/3735685427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8103660 0x7f19b810bb50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009900 tx=0x7f19ac02a480 comp rx=0 tx=0).stop 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 -- 172.21.15.28:0/3735685427 shutdown_connections 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 --2- 172.21.15.28:0/3735685427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 0x7f19b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 --2- 172.21.15.28:0/3735685427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8103660 0x7f19b810bb50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 --2- 172.21.15.28:0/3735685427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8102d40 0x7f19b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 -- 172.21.15.28:0/3735685427 >> 172.21.15.28:0/3735685427 conn(0x7f19b80fc7a0 msgr2=0x7f19b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 -- 172.21.15.28:0/3735685427 shutdown_connections 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.957+0000 7f19be5b5640 1 -- 172.21.15.28:0/3735685427 wait complete. 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 Processor -- start 2024-09-19T09:32:51.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 -- start start 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8103660 0x7f19b8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 0x7f19b819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8111f70 con 0x7f19b8102d40 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8111df0 con 0x7f19b810c170 2024-09-19T09:32:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.958+0000 7f19be5b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81120f0 con 0x7f19b8103660 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54282/0 (socket says 172.21.15.28:54282) 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 -- 172.21.15.28:0/1332215991 learned_addr learned my addr 172.21.15.28:0/1332215991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8103660 0x7f19b8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 0x7f19b819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 -- 172.21.15.28:0/1332215991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8103660 msgr2=0x7f19b8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8103660 0x7f19b8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 -- 172.21.15.28:0/1332215991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 msgr2=0x7f19b819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 0x7f19b819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 -- 172.21.15.28:0/1332215991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b819d400 con 0x7f19b8102d40 2024-09-19T09:32:51.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d770 tx=0x7f19a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b57fa640 1 -- 172.21.15.28:0/1332215991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8017070 con 0x7f19b8102d40 2024-09-19T09:32:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b57fa640 1 -- 172.21.15.28:0/1332215991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a8005500 con 0x7f19b8102d40 2024-09-19T09:32:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b819d6f0 con 0x7f19b8102d40 2024-09-19T09:32:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19b57fa640 1 -- 172.21.15.28:0/1332215991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8004e00 con 0x7f19b8102d40 2024-09-19T09:32:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.959+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b81a4fd0 con 0x7f19b8102d40 2024-09-19T09:32:51.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19b57fa640 1 -- 172.21.15.28:0/1332215991 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f19a8004fa0 con 0x7f19b8102d40 2024-09-19T09:32:51.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 Processor -- start 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 -- start start 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d830c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d8423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c89d797090 con 0x55c89d8311d0 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89d73ea70 con 0x55c89d8308b0 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19c098e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c89d793050 con 0x55c89d83ffc0 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.962+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54294/0 (socket says 172.21.15.28:54294) 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 -- 172.21.15.28:0/433247792 learned_addr learned my addr 172.21.15.28:0/433247792 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:51.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19bcb52640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d830c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b57fa640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 0x7f19a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b77fe640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d8423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 0x7f19a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b57fa640 1 -- 172.21.15.28:0/1332215991 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f19a80cf440 con 0x7f19b8102d40 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 -- 172.21.15.28:0/433247792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 msgr2=0x55c89d8423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d8423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 -- 172.21.15.28:0/433247792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 msgr2=0x55c89d830c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d830c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 -- 172.21.15.28:0/433247792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89d74b130 con 0x55c89d8311d0 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19bcb52640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d830c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b77fe640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d8423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19b7fff640 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19a80983b0 tx=0x7f19a8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3588214c265306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 0x7f19a00a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005fd0 tx=0x7f19b0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f199e7fc640 1 -- 172.21.15.28:0/433247792 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8020030 con 0x55c89d8311d0 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.963+0000 7f199e7fc640 1 -- 172.21.15.28:0/433247792 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a809b9e0 con 0x55c89d8311d0 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f199e7fc640 1 -- 172.21.15.28:0/433247792 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a80860e0 con 0x55c89d8311d0 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 -- 172.21.15.28:0/433247792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 msgr2=0x55c89d83fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19a80983b0 tx=0x7f19a8009c30 comp rx=0 tx=0).stop 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 -- 172.21.15.28:0/433247792 shutdown_connections 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d8423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8311d0 0x55c89d83fa80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 --2- 172.21.15.28:0/433247792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d830c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 -- 172.21.15.28:0/433247792 >> 172.21.15.28:0/433247792 conn(0x55c89d82cf60 msgr2=0x55c89d82d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 -- 172.21.15.28:0/433247792 shutdown_connections 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.964+0000 7f19c098e740 1 -- 172.21.15.28:0/433247792 wait complete. 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 -- start start 2024-09-19T09:32:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 Processor -- start 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d83cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d83d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c89d842b30 con 0x55c89d8308b0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89d751f20 con 0x55c89d8311d0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c89d842cb0 con 0x55c89d83ffc0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19c098e740 10 client.0 _ll_register_callbacks cb 0x55c89d71b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54296/0 (socket says 172.21.15.28:54296) 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19b7fff640 1 -- 172.21.15.28:0/1136859488 learned_addr learned my addr 172.21.15.28:0/1136859488 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.965+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d83cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d83d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 -- 172.21.15.28:0/1136859488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 msgr2=0x55c89d83d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d83d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 -- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 msgr2=0x55c89d83cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d83cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 -- 172.21.15.28:0/1136859488 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89d5082b0 con 0x55c89d8308b0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d83d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19a809b970 tx=0x7f19a80d2ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6cbd80b06c24c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8020070 con 0x55c89d8308b0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f19a8009cc0 con 0x55c89d8308b0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a809b9e0 con 0x55c89d8308b0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19a80b1e60 con 0x55c89d8308b0 2024-09-19T09:32:51.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19c098e740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89d84e4e0 con 0x55c89d8308b0 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.966+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a80afe50 con 0x55c89d8308b0 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19a80cf0d0 con 0x55c89d8308b0 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 20 client.15408 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8506b0 0x55c89d851b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 4 client.15408 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89d7a4600 con 0x55c89d8506b0 2024-09-19T09:32:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8506b0 0x55c89d851b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8506b0 0x55c89d851b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=239dc4318700b2a4 server_cookie=e6dd7c1e128cf99e in_seq=0 out_seq=0 2024-09-19T09:32:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.967+0000 7f19977fe640 10 client.15408 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19977fe640 1 -- 172.21.15.28:0/1136859488 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c89d7a4600 con 0x55c89d8506b0 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19977fe640 10 client.15408 handle_command_reply: tid=0 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 client.15408 shutdown 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 msgr2=0x55c89d83f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19a809b970 tx=0x7f19a80d2ba0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 shutdown_connections 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8506b0 0x55c89d851b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d83cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d83d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 --2- 172.21.15.28:0/1136859488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d83f8c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 >> 172.21.15.28:0/1136859488 conn(0x55c89d82cf60 msgr2=0x55c89d832cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.968+0000 7f19977fe640 0 client.15408 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.969+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 shutdown_connections 2024-09-19T09:32:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.969+0000 7f19c098e740 1 -- 172.21.15.28:0/1136859488 wait complete. 2024-09-19T09:32:51.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.969+0000 7f19c098e740 20 client.15408 trim_cache size 0 max 16384 2024-09-19T09:32:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 Processor -- start 2024-09-19T09:32:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 -- start start 2024-09-19T09:32:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d8506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 0x55c89d827d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c89d84fe60 con 0x55c89d83ffc0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89d850530 con 0x55c89d8308b0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19c098e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c89d850c00 con 0x55c89d8311d0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54306/0 (socket says 172.21.15.28:54306) 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 -- 172.21.15.28:0/1928121038 learned_addr learned my addr 172.21.15.28:0/1928121038 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19b7fff640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d8506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 0x55c89d827d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 -- 172.21.15.28:0/1928121038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 msgr2=0x55c89d827d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 0x55c89d827d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 -- 172.21.15.28:0/1928121038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 msgr2=0x55c89d8506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d8506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 -- 172.21.15.28:0/1928121038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89d83fc30 con 0x55c89d83ffc0 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19b77fe640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 0x55c89d827d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.987+0000 7f19bcb52640 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19b00147a0 tx=0x7f19b0009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23054ff7d7756b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f199dffb640 1 -- 172.21.15.28:0/1928121038 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001a070 con 0x55c89d83ffc0 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f199dffb640 1 -- 172.21.15.28:0/1928121038 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0025030 con 0x55c89d83ffc0 2024-09-19T09:32:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f199dffb640 1 -- 172.21.15.28:0/1928121038 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001eb10 con 0x55c89d83ffc0 2024-09-19T09:32:51.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 -- 172.21.15.28:0/1928121038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 msgr2=0x55c89d83eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19b00147a0 tx=0x7f19b0009ab0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 -- 172.21.15.28:0/1928121038 shutdown_connections 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d83ffc0 0x55c89d83eb80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d8311d0 0x55c89d827d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 --2- 172.21.15.28:0/1928121038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8308b0 0x55c89d8506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 -- 172.21.15.28:0/1928121038 >> 172.21.15.28:0/1928121038 conn(0x55c89d82cf60 msgr2=0x55c89d8382d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 -- 172.21.15.28:0/1928121038 shutdown_connections 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.988+0000 7f19c098e740 1 -- 172.21.15.28:0/1928121038 wait complete. 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 -- start start 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 Processor -- start 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d839980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d858d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c89d8519a0 con 0x55c89d8308b0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c89d8512d0 con 0x55c89d8311d0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c89d852070 con 0x55c89d83ffc0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19c098e740 10 client.0 _ll_register_callbacks cb 0x7f19b0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.989+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54322/0 (socket says 172.21.15.28:54322) 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 -- 172.21.15.28:0/3469002074 learned_addr learned my addr 172.21.15.28:0/3469002074 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b77fe640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d839980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19bcb52640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d858d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 -- 172.21.15.28:0/3469002074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 msgr2=0x55c89d858d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d858d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 -- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 msgr2=0x55c89d839980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d839980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 -- 172.21.15.28:0/3469002074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89d8592b0 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19b7fff640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19a80cd3a0 tx=0x7f19a80056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d93bde206e0df80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8020070 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f19a8097930 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b0025030 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19a80a45a0 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19c098e740 10 client.15414 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89d85a480 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.990+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0025030 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19a8098050 con 0x55c89d8308b0 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 10 client.15414 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 20 client.15414 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8beff0 0x55c89d8bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 4 client.15414 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "status"}) -- 0x55c89d79eb10 con 0x55c89d8beff0 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19b77fe640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8beff0 0x55c89d8bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19b77fe640 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8beff0 0x55c89d8bdd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78a91ce9eaafdda7 server_cookie=bcf7f9968468f807 in_seq=0 out_seq=0 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.991+0000 7f19977fe640 10 client.15414 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19977fe640 1 -- 172.21.15.28:0/3469002074 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c89d79eb10 con 0x55c89d8beff0 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19977fe640 10 client.15414 handle_command_reply: tid=0 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 client.15414 shutdown 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 msgr2=0x55c89d839440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19a80cd3a0 tx=0x7f19a80056b0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 shutdown_connections 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55c89d8beff0 0x55c89d8bdd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c89d8311d0 0x55c89d839980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19977fe640 0 client.15414 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c89d83ffc0 0x55c89d858d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 --2- 172.21.15.28:0/3469002074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c89d8308b0 0x55c89d839440 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 >> 172.21.15.28:0/3469002074 conn(0x55c89d82cf60 msgr2=0x55c89d827d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 shutdown_connections 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 1 -- 172.21.15.28:0/3469002074 wait complete. 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.992+0000 7f19c098e740 20 client.15414 trim_cache size 0 max 16384 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:51.999 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 4, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24455, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 118.81460327400001, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 27, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 149.70539796700001 2024-09-19T09:32:52.000 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 msgr2=0x7f19a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 0x7f19a00a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005fd0 tx=0x7f19b0004ab0 comp rx=0 tx=0).stop 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 msgr2=0x7f19b8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d770 tx=0x7f19a800dc30 comp rx=0 tx=0).stop 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 shutdown_connections 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f19a00a32d0 0x7f19a00a5790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b810c170 0x7f19b819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19b8103660 0x7f19b8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:52.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 --2- 172.21.15.28:0/1332215991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b8102d40 0x7f19b8198450 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:52.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 >> 172.21.15.28:0/1332215991 conn(0x7f19b80fc7a0 msgr2=0x7f19b80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:52.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 shutdown_connections 2024-09-19T09:32:52.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:51.993+0000 7f19be5b5640 1 -- 172.21.15.28:0/1332215991 wait complete. 2024-09-19T09:32:52.690 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:52 smithi028 ceph-mon[30972]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:32:52.720 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:32:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:52 smithi078 ceph-mon[35619]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:32:52.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:52 smithi038 ceph-mon[35271]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:32:52.976 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:53.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 -- 172.21.15.28:0/605134588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84102ef0 msgr2=0x7f6c8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/605134588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84102ef0 0x7f6c8410da40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009ca0 tx=0x7f6c7c02a950 comp rx=0 tx=0).stop 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 -- 172.21.15.28:0/605134588 shutdown_connections 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/605134588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84102ef0 0x7f6c8410da40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/605134588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 0x7f6c841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/605134588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84100e20 0x7f6c84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 -- 172.21.15.28:0/605134588 >> 172.21.15.28:0/605134588 conn(0x7f6c840fc7d0 msgr2=0x7f6c840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 -- 172.21.15.28:0/605134588 shutdown_connections 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.442+0000 7f6c89cf7640 1 -- 172.21.15.28:0/605134588 wait complete. 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 Processor -- start 2024-09-19T09:32:53.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 -- start start 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 0x7f6c841989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 0x7f6c8419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84110f80 con 0x7f6c84100e20 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84110e00 con 0x7f6c84102ef0 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c89cf7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84111100 con 0x7f6c84102570 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.443+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43466/0 (socket says 172.21.15.28:43466) 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 -- 172.21.15.28:0/6482943 learned_addr learned my addr 172.21.15.28:0/6482943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 0x7f6c841989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c83fff640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 0x7f6c8419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 -- 172.21.15.28:0/6482943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 msgr2=0x7f6c841989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 0x7f6c841989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 -- 172.21.15.28:0/6482943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 msgr2=0x7f6c8419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 0x7f6c8419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 -- 172.21.15.28:0/6482943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8419d410 con 0x7f6c84100e20 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c83fff640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 0x7f6c8419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c837fe640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400d950 tx=0x7f6c7400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c80ff9640 1 -- 172.21.15.28:0/6482943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74017070 con 0x7f6c84100e20 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c80ff9640 1 -- 172.21.15.28:0/6482943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7400de80 con 0x7f6c84100e20 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8419d700 con 0x7f6c84100e20 2024-09-19T09:32:53.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c80ff9640 1 -- 172.21.15.28:0/6482943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400f620 con 0x7f6c84100e20 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.444+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a4fe0 con 0x7f6c84100e20 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.446+0000 7f6c80ff9640 1 -- 172.21.15.28:0/6482943 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6c74009c30 con 0x7f6c84100e20 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 Processor -- start 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 -- start start 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 0x55f3943fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f394357090 con 0x55f3943f09d0 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3942fea70 con 0x55f3943fbbb0 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f394353050 con 0x55f3943f12f0 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43476/0 (socket says 172.21.15.28:43476) 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 -- 172.21.15.28:0/2124710667 learned_addr learned my addr 172.21.15.28:0/2124710667 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c80ff9640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 0x7f6c6c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c837fe640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 0x55f3943fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 -- 172.21.15.28:0/2124710667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 msgr2=0x55f3943fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 0x55f3943fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 -- 172.21.15.28:0/2124710667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 msgr2=0x55f3943fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.447+0000 7f6c83fff640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943fdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c83fff640 1 -- 172.21.15.28:0/2124710667 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39430b130 con 0x55f3943f09d0 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c837fe640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 0x55f3943fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c80ff9640 1 -- 172.21.15.28:0/6482943 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6c740cf660 con 0x7f6c84100e20 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c83fff640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 0x7f6c6c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c83fff640 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c004af0 tx=0x7f6c7c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32c4771d35e6fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c69ffb640 1 -- 172.21.15.28:0/2124710667 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c040040 con 0x55f3943f09d0 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c69ffb640 1 -- 172.21.15.28:0/2124710667 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c003cb0 con 0x55f3943f09d0 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.448+0000 7f6c69ffb640 1 -- 172.21.15.28:0/2124710667 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c034070 con 0x55f3943f09d0 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c83fff640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 0x7f6c6c0a5810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c04b000 tx=0x7f6c7c04c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/2124710667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 msgr2=0x55f3943f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c004af0 tx=0x7f6c7c009930 comp rx=0 tx=0).stop 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/2124710667 shutdown_connections 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f12f0 0x55f3943fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/2124710667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943f0db0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/2124710667 >> 172.21.15.28:0/2124710667 conn(0x55f3943ecf60 msgr2=0x55f3943ed340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/2124710667 shutdown_connections 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/2124710667 wait complete. 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 -- start start 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.449+0000 7f6c8c0d0740 1 Processor -- start 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943fbbb0 0x55f3943f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f3943fe780 con 0x55f3943f09d0 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f394311f20 con 0x55f3943f12f0 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f3943fe900 con 0x55f3943fbbb0 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c8c0d0740 10 client.0 _ll_register_callbacks cb 0x55f3942db030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43488/0 (socket says 172.21.15.28:43488) 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 -- 172.21.15.28:0/3288408311 learned_addr learned my addr 172.21.15.28:0/3288408311 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c83fff640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943fbbb0 0x55f3943f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 -- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943fbbb0 msgr2=0x55f3943f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943fbbb0 0x55f3943f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 -- 172.21.15.28:0/3288408311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 msgr2=0x55f3943f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 -- 172.21.15.28:0/3288408311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3940c82b0 con 0x55f3943f09d0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c837fe640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c740cd070 tx=0x7f6c7409bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa194d453de2506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.450+0000 7f6c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74020070 con 0x55f3943f09d0 2024-09-19T09:32:53.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6c740860e0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c003cb0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c740a47c0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f39440a0c0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c003cb0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c7409aaf0 con 0x55f3943f09d0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 10 client.15426 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 20 client.15426 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39440c6e0 0x55f39440db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 4 client.15426 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.451+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f394364600 con 0x55f39440c6e0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.452+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39440c6e0 0x55f39440db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.452+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39440c6e0 0x55f39440db50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3db2b37247b6f2b server_cookie=81a722a9810f1208 in_seq=0 out_seq=0 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.452+0000 7f6c62ffd640 10 client.15426 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c62ffd640 1 -- 172.21.15.28:0/3288408311 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f394364600 con 0x55f39440c6e0 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c62ffd640 10 client.15426 handle_command_reply: tid=0 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 client.15426 shutdown 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 msgr2=0x55f3943fb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c740cd070 tx=0x7f6c7409bb50 comp rx=0 tx=0).stop 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 shutdown_connections 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39440c6e0 0x55f39440db50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943fbbb0 0x55f3943f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/3288408311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f09d0 0x55f3943fb3f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 >> 172.21.15.28:0/3288408311 conn(0x55f3943ecf60 msgr2=0x55f3943f2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c62ffd640 0 client.15426 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 shutdown_connections 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/3288408311 wait complete. 2024-09-19T09:32:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.453+0000 7f6c8c0d0740 20 client.15426 trim_cache size 0 max 16384 2024-09-19T09:32:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 Processor -- start 2024-09-19T09:32:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 -- start start 2024-09-19T09:32:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f39440c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f39440be90 con 0x55f3943fbbb0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f39440c560 con 0x55f3943f12f0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f39440cc30 con 0x55f3943f09d0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43494/0 (socket says 172.21.15.28:43494) 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c83fff640 1 -- 172.21.15.28:0/1639615618 learned_addr learned my addr 172.21.15.28:0/1639615618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.471+0000 7f6c837fe640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f39440c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 -- 172.21.15.28:0/1639615618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 msgr2=0x55f39440c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f39440c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 -- 172.21.15.28:0/1639615618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 msgr2=0x55f3943ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 -- 172.21.15.28:0/1639615618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3943fb760 con 0x55f3943fbbb0 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c83fff640 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c02a990 tx=0x7f6c7c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f34162a92450 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c697fa640 1 -- 172.21.15.28:0/1639615618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c040070 con 0x55f3943fbbb0 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c697fa640 1 -- 172.21.15.28:0/1639615618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c030440 con 0x55f3943fbbb0 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.472+0000 7f6c697fa640 1 -- 172.21.15.28:0/1639615618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c034040 con 0x55f3943fbbb0 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/1639615618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 msgr2=0x55f394416220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c02a990 tx=0x7f6c7c04f000 comp rx=0 tx=0).stop 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/1639615618 shutdown_connections 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943f12f0 0x55f3943ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943fbbb0 0x55f394416220 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/1639615618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f39440c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/1639615618 >> 172.21.15.28:0/1639615618 conn(0x55f3943ecf60 msgr2=0x55f3943e7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/1639615618 shutdown_connections 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/1639615618 wait complete. 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.473+0000 7f6c8c0d0740 1 -- start start 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 Processor -- start 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f394416680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f39440d9d0 con 0x55f3943f12f0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f39440d300 con 0x55f3943fbbb0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f39440e0a0 con 0x55f3943f09d0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c8c0d0740 10 client.0 _ll_register_callbacks cb 0x7f6c7409b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43500/0 (socket says 172.21.15.28:43500) 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 -- 172.21.15.28:0/55107155 learned_addr learned my addr 172.21.15.28:0/55107155 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c83fff640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c837fe640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f394416680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 -- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 msgr2=0x55f394416680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f394416680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 -- 172.21.15.28:0/55107155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 msgr2=0x55f3943f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c82ffd640 1 -- 172.21.15.28:0/55107155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3943f6830 con 0x55f3943f12f0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.474+0000 7f6c83fff640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c82ffd640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdb4cc84fb19a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78005220 con 0x55f3943f12f0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f6c780053c0 con 0x55f3943f12f0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c030440 con 0x55f3943f12f0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c7802c050 con 0x55f3943f12f0 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 10 client.15432 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f394479fb0 con 0x55f3943f12f0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c030440 con 0x55f3943f12f0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c7801d720 con 0x55f3943f12f0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 10 client.15432 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 10 client.15432 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 20 client.15432 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.475+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39447eaf0 0x55f39447d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c8c0d0740 4 client.15432 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "status"}) -- 0x55f39435eb10 con 0x55f39447eaf0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c837fe640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39447eaf0 0x55f39447d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c837fe640 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39447eaf0 0x55f39447d800 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b48b72998c0d460 server_cookie=3d9b06ada33bbaca in_seq=0 out_seq=0 2024-09-19T09:32:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c62ffd640 10 client.15432 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c62ffd640 1 -- 172.21.15.28:0/55107155 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f39435eb10 con 0x55f39447eaf0 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c62ffd640 10 client.15432 handle_command_reply: tid=0 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c8c0d0740 1 client.15432 shutdown 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 msgr2=0x55f3943f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.476+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78013830 comp rx=0 tx=0).stop 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 shutdown_connections 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f39447eaf0 0x55f39447d800 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3943fbbb0 0x55f3943f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f3943f12f0 0x55f3943f5a10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 --2- 172.21.15.28:0/55107155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f3943f09d0 0x55f394416680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 >> 172.21.15.28:0/55107155 conn(0x55f3943ecf60 msgr2=0x55f39440c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c62ffd640 0 client.15432 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 shutdown_connections 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 1 -- 172.21.15.28:0/55107155 wait complete. 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c8c0d0740 20 client.15432 trim_cache size 0 max 16384 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 3, 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24472, 2024-09-19T09:32:53.493 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 121.304210918, 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 27, 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 156.13791868999999 2024-09-19T09:32:53.494 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 msgr2=0x7f6c6c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 0x7f6c6c0a5810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c04b000 tx=0x7f6c7c04c000 comp rx=0 tx=0).stop 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 msgr2=0x7f6c84198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400d950 tx=0x7f6c7400dd50 comp rx=0 tx=0).stop 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 shutdown_connections 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6c6c0a3350 0x7f6c6c0a5810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c84102ef0 0x7f6c8419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102570 0x7f6c841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 --2- 172.21.15.28:0/6482943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c84100e20 0x7f6c84198460 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:53.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.477+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 >> 172.21.15.28:0/6482943 conn(0x7f6c840fc7d0 msgr2=0x7f6c8406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:53.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.478+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 shutdown_connections 2024-09-19T09:32:53.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:53.478+0000 7f6c89cf7640 1 -- 172.21.15.28:0/6482943 wait complete. 2024-09-19T09:32:54.182 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:32:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:54.733 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:54 smithi028 ceph-mon[30972]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:32:54.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:54 smithi078 ceph-mon[35619]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 -- 172.21.15.28:0/966613051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 msgr2=0x7fdbb8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 --2- 172.21.15.28:0/966613051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8101200 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802a480 comp rx=0 tx=0).stop 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 -- 172.21.15.28:0/966613051 shutdown_connections 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 --2- 172.21.15.28:0/966613051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102ef0 0x7fdbb810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 --2- 172.21.15.28:0/966613051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102570 0x7fdbb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 --2- 172.21.15.28:0/966613051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8101200 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 -- 172.21.15.28:0/966613051 >> 172.21.15.28:0/966613051 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 -- 172.21.15.28:0/966613051 shutdown_connections 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.902+0000 7fdbbd436640 1 -- 172.21.15.28:0/966613051 wait complete. 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.903+0000 7fdbbd436640 1 Processor -- start 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.903+0000 7fdbbd436640 1 -- start start 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102570 0x7fdbb8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8111020 con 0x7fdbb8100e20 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8110ea0 con 0x7fdbb8102570 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbbd436640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81111a0 con 0x7fdbb8102ef0 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43528/0 (socket says 172.21.15.28:43528) 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2181649896 learned_addr learned my addr 172.21.15.28:0/2181649896 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102570 0x7fdbb8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2181649896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102ef0 msgr2=0x7fdbb819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2181649896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102570 msgr2=0x7fdbb8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102570 0x7fdbb8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.904+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2181649896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb819c530 con 0x7fdbb8100e20 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdba8006010 tx=0x7fdba8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdb9ffff640 1 -- 172.21.15.28:0/2181649896 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba803f070 con 0x7fdbb8100e20 2024-09-19T09:32:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdb9ffff640 1 -- 172.21.15.28:0/2181649896 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8031e30 con 0x7fdbb8100e20 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdb9ffff640 1 -- 172.21.15.28:0/2181649896 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8035030 con 0x7fdbb8100e20 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb819c7c0 con 0x7fdbb8100e20 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.905+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a4100 con 0x7fdbb8100e20 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.907+0000 7fdb9ffff640 1 -- 172.21.15.28:0/2181649896 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fdba8042070 con 0x7fdbb8100e20 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.907+0000 7fdbbf80f740 1 Processor -- start 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 -- start start 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f9011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56226f858090 con 0x56226f8fedc0 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56226f7ffa70 con 0x56226f8f18f0 2024-09-19T09:32:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56226f854050 con 0x56226f8f2210 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:51838/0 (socket says 172.21.15.28:51838) 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/4111682173 learned_addr learned my addr 172.21.15.28:0/4111682173 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdb9ffff640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 0x7fdba00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.908+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f9011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/4111682173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 msgr2=0x56226f8f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/4111682173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 msgr2=0x56226f9011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f9011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/4111682173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226f80c130 con 0x56226f8f2210 2024-09-19T09:32:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdb9ffff640 1 -- 172.21.15.28:0/2181649896 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdba80f03c0 con 0x7fdbb8100e20 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 0x7fdba00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b8d10 tx=0x7fdba804b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad07f9c51941106 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdb9d7fa640 1 -- 172.21.15.28:0/4111682173 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80bcfb0 con 0x56226f8f2210 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdb9d7fa640 1 -- 172.21.15.28:0/4111682173 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80bc8b0 con 0x56226f8f2210 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.909+0000 7fdb9d7fa640 1 -- 172.21.15.28:0/4111682173 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80c0c60 con 0x56226f8f2210 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 -- 172.21.15.28:0/4111682173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 msgr2=0x56226f8fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b8d10 tx=0x7fdba804b000 comp rx=0 tx=0).stop 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 0x7fdba00a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009eb0 tx=0x7fdbb000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 -- 172.21.15.28:0/4111682173 shutdown_connections 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f9011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8fe880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/4111682173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 -- 172.21.15.28:0/4111682173 >> 172.21.15.28:0/4111682173 conn(0x56226f8edf60 msgr2=0x56226f8ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 -- 172.21.15.28:0/4111682173 shutdown_connections 2024-09-19T09:32:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.910+0000 7fdbbf80f740 1 -- 172.21.15.28:0/4111682173 wait complete. 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 -- start start 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 Processor -- start 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 0x56226f8fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f2210 0x56226f8f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56226f8f4610 con 0x56226f8fedc0 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56226f812f20 con 0x56226f8f2210 2024-09-19T09:32:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56226f8f4790 con 0x56226f8f18f0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbbf80f740 10 client.0 _ll_register_callbacks cb 0x56226f7dc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 0x56226f8fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43538/0 (socket says 172.21.15.28:43538) 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbb77fe640 1 -- 172.21.15.28:0/2218067125 learned_addr learned my addr 172.21.15.28:0/2218067125 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.911+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f2210 0x56226f8f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2218067125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f2210 msgr2=0x56226f8f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f2210 0x56226f8f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2218067125 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 msgr2=0x56226f8fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 -- 172.21.15.28:0/2218067125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226f5c92b0 con 0x56226f8f18f0 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 0x56226f8fe7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b9280 tx=0x7fdba8034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db3928c59e89f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80bcfb0 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba80bc8b0 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdba803b620 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdba80c5520 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbbf80f740 10 client.25069 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56226f8f4610 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.912+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80bc8b0 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdba8084170 con 0x56226f8f18f0 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 10 client.25069 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 10 client.25069 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 20 client.25069 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f9115b0 0x56226f912a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 4 client.25069 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56226f865600 con 0x56226f9115b0 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f9115b0 0x56226f912a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f9115b0 0x56226f912a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80f531b2bf520b9 server_cookie=86221ffb3c3d04af in_seq=0 out_seq=0 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.913+0000 7fdb9aff5640 10 client.25069 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdb9aff5640 1 -- 172.21.15.28:0/2218067125 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56226f865600 con 0x56226f9115b0 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdb9aff5640 10 client.25069 handle_command_reply: tid=0 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 client.25069 shutdown 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 msgr2=0x56226f8fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 0x56226f8fe7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b9280 tx=0x7fdba8034000 comp rx=0 tx=0).stop 2024-09-19T09:32:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 shutdown_connections 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f9115b0 0x56226f912a20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f2210 0x56226f8f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f8fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdb9aff5640 0 client.25069 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2218067125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f18f0 0x56226f8fe7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 >> 172.21.15.28:0/2218067125 conn(0x56226f8edf60 msgr2=0x56226f8f3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.914+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 shutdown_connections 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.915+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2218067125 wait complete. 2024-09-19T09:32:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.915+0000 7fdbbf80f740 20 client.25069 trim_cache size 0 max 16384 2024-09-19T09:32:54.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:54 smithi038 ceph-mon[35271]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.932+0000 7fdbbf80f740 1 Processor -- start 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 -- start start 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f912a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56226f910d60 con 0x56226f8fedc0 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56226f911430 con 0x56226f8f18f0 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56226f911b00 con 0x56226f8f2210 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43542/0 (socket says 172.21.15.28:43542) 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 -- 172.21.15.28:0/2318815950 learned_addr learned my addr 172.21.15.28:0/2318815950 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f912a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 -- 172.21.15.28:0/2318815950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 msgr2=0x56226f8f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 -- 172.21.15.28:0/2318815950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 msgr2=0x56226f912a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f912a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.933+0000 7fdbb77fe640 1 -- 172.21.15.28:0/2318815950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226f8eefa0 con 0x56226f8fedc0 2024-09-19T09:32:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.934+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0015c00 tx=0x7fdbb0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb8c8b946247674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.934+0000 7fdb9cff9640 1 -- 172.21.15.28:0/2318815950 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0025070 con 0x56226f8fedc0 2024-09-19T09:32:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.934+0000 7fdb9cff9640 1 -- 172.21.15.28:0/2318815950 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb001c040 con 0x56226f8fedc0 2024-09-19T09:32:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.934+0000 7fdb9cff9640 1 -- 172.21.15.28:0/2318815950 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb001b040 con 0x56226f8fedc0 2024-09-19T09:32:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.934+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2318815950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 msgr2=0x56226f91ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0015c00 tx=0x7fdbb0012000 comp rx=0 tx=0).stop 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2318815950 shutdown_connections 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8fedc0 0x56226f91ad00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8f2210 0x56226f8f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/2318815950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f912a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2318815950 >> 172.21.15.28:0/2318815950 conn(0x56226f8edf60 msgr2=0x56226f8f49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2318815950 shutdown_connections 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- 172.21.15.28:0/2318815950 wait complete. 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- start start 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 Processor -- start 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8fedc0 0x56226f8f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56226f9128a0 con 0x56226f8f2210 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56226f9121d0 con 0x56226f8f18f0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56226f912f70 con 0x56226f8fedc0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbbf80f740 10 client.0 _ll_register_callbacks cb 0x7fdba80bc480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43548/0 (socket says 172.21.15.28:43548) 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.935+0000 7fdbb67fc640 1 -- 172.21.15.28:0/1549398304 learned_addr learned my addr 172.21.15.28:0/1549398304 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb77fe640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8fedc0 0x56226f8f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 -- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8fedc0 msgr2=0x56226f8f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8fedc0 0x56226f8f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 -- 172.21.15.28:0/1549398304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 msgr2=0x56226f8f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 -- 172.21.15.28:0/1549398304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226f8f6a60 con 0x56226f8f2210 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbb67fc640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x56226f8f9360 tx=0x7fdbac0157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d8e89627ba5e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdb9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac025470 con 0x56226f8f2210 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fdbac025610 con 0x56226f8f2210 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb001c040 con 0x56226f8f2210 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbac02c070 con 0x56226f8f2210 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbbf80f740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.936+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56226f90cb00 con 0x56226f8f2210 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb001c040 con 0x56226f8f2210 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdbac030890 con 0x56226f8f2210 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 10 client.15450 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 20 client.15450 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f980250 0x56226f97ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 4 client.15450 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "status"}) -- 0x56226f85fb10 con 0x56226f980250 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.937+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f980250 0x56226f97ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdb9aff5640 10 client.15450 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbb6ffd640 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f980250 0x56226f97ef60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cd9a96258990f47 server_cookie=835fe7158cfb1f04 in_seq=0 out_seq=0 2024-09-19T09:32:54.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdb9aff5640 1 -- 172.21.15.28:0/1549398304 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56226f85fb10 con 0x56226f980250 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdb9aff5640 10 client.15450 handle_command_reply: tid=0 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 client.15450 shutdown 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 msgr2=0x56226f8f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x56226f8f9360 tx=0x7fdbac0157f0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 shutdown_connections 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56226f980250 0x56226f97ef60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56226f8fedc0 0x56226f8f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56226f8f2210 0x56226f8f5d20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 --2- 172.21.15.28:0/1549398304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56226f8f18f0 0x56226f8f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 >> 172.21.15.28:0/1549398304 conn(0x56226f8edf60 msgr2=0x56226f8f50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdb9aff5640 0 client.15450 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 shutdown_connections 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 1 -- 172.21.15.28:0/1549398304 wait complete. 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.938+0000 7fdbbf80f740 20 client.15450 trim_cache size 0 max 16384 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 2, 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24478, 2024-09-19T09:32:54.946 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 123.783280183, 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 27, 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 155.17054780300001 2024-09-19T09:32:54.947 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 msgr2=0x7fdba00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 0x7fdba00a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009eb0 tx=0x7fdbb000d040 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 msgr2=0x7fdbb8197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdba8006010 tx=0x7fdba8003c80 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 shutdown_connections 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdba00a32d0 0x7fdba00a5790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102570 0x7fdbb8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 --2- 172.21.15.28:0/2181649896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbb8100e20 0x7fdbb8197580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 >> 172.21.15.28:0/2181649896 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 shutdown_connections 2024-09-19T09:32:54.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:54.939+0000 7fdbbd436640 1 -- 172.21.15.28:0/2181649896 wait complete. 2024-09-19T09:32:55.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:32:55.968 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 -- 172.21.15.28:0/2240608046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc100ca0 msgr2=0x7febfc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 --2- 172.21.15.28:0/2240608046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc100ca0 0x7febfc106840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7febe8009930 tx=0x7febe802a420 comp rx=0 tx=0).stop 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 -- 172.21.15.28:0/2240608046 shutdown_connections 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 --2- 172.21.15.28:0/2240608046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 0x7febfc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 --2- 172.21.15.28:0/2240608046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc100ca0 0x7febfc106840 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 --2- 172.21.15.28:0/2240608046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc1002f0 0x7febfc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 -- 172.21.15.28:0/2240608046 >> 172.21.15.28:0/2240608046 conn(0x7febfc077210 msgr2=0x7febfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 -- 172.21.15.28:0/2240608046 shutdown_connections 2024-09-19T09:32:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.417+0000 7fec02d35640 1 -- 172.21.15.28:0/2240608046 wait complete. 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.418+0000 7fec02d35640 1 Processor -- start 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.418+0000 7fec02d35640 1 -- start start 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 0x7febfc1a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 0x7febfc10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7febfc111f60 con 0x7febfc1002f0 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7febfc111de0 con 0x7febfc106f50 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1120e0 con 0x7febfc100ca0 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43578/0 (socket says 172.21.15.28:43578) 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 -- 172.21.15.28:0/4293685656 learned_addr learned my addr 172.21.15.28:0/4293685656 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec012d2640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 0x7febfc10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7febfbfff640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 0x7febfc1a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 -- 172.21.15.28:0/4293685656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 msgr2=0x7febfc1a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 0x7febfc1a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 -- 172.21.15.28:0/4293685656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 msgr2=0x7febfc10dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 0x7febfc10dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 -- 172.21.15.28:0/4293685656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc10e450 con 0x7febfc1002f0 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7febfbfff640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 0x7febfc1a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec00ad1640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7febf000c920 tx=0x7febf000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7febf9ffb640 1 -- 172.21.15.28:0/4293685656 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0007c70 con 0x7febfc1002f0 2024-09-19T09:32:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc10e740 con 0x7febfc1002f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.419+0000 7febf9ffb640 1 -- 172.21.15.28:0/4293685656 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf0007e10 con 0x7febfc1002f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.420+0000 7febf9ffb640 1 -- 172.21.15.28:0/4293685656 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0005260 con 0x7febfc1002f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.420+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc1ab640 con 0x7febfc1002f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7febf9ffb640 1 -- 172.21.15.28:0/4293685656 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7febf0005400 con 0x7febfc1002f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 Processor -- start 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 -- start start 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6749d0 0x55a33f674db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 0x55a33f681fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a33f5db090 con 0x55a33f6752f0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a33f582a70 con 0x55a33f67fbb0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.423+0000 7fec0510e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a33f5d7050 con 0x55a33f6749d0 2024-09-19T09:32:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7febf9ffb640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 0x7febec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43592/0 (socket says 172.21.15.28:43592) 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 -- 172.21.15.28:0/3309633766 learned_addr learned my addr 172.21.15.28:0/3309633766 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec012d2640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6749d0 0x55a33f674db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec012d2640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 0x7febec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7febfbfff640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 0x55a33f681fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7febf9ffb640 1 -- 172.21.15.28:0/4293685656 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7febf00cfcf0 con 0x7febfc1002f0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 -- 172.21.15.28:0/3309633766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6749d0 msgr2=0x55a33f674db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6749d0 0x55a33f674db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 -- 172.21.15.28:0/3309633766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 msgr2=0x55a33f681fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 0x55a33f681fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 -- 172.21.15.28:0/3309633766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33f58f130 con 0x55a33f6752f0 2024-09-19T09:32:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7febfbfff640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 0x55a33f681fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec00ad1640 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febf009c0c0 tx=0x7febf000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c0b1b3eeb5ede server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7fec012d2640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 0x7febec0a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7febf400bfd0 tx=0x7febf4005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.424+0000 7febe6ffd640 1 -- 172.21.15.28:0/3309633766 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00056f0 con 0x55a33f6752f0 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7febe6ffd640 1 -- 172.21.15.28:0/3309633766 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf00aa420 con 0x55a33f6752f0 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7febe6ffd640 1 -- 172.21.15.28:0/3309633766 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00cd8c0 con 0x55a33f6752f0 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 -- 172.21.15.28:0/3309633766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 msgr2=0x55a33f67f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febf009c0c0 tx=0x7febf000ff30 comp rx=0 tx=0).stop 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 -- 172.21.15.28:0/3309633766 shutdown_connections 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f67fbb0 0x55a33f681fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f67f670 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 --2- 172.21.15.28:0/3309633766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6749d0 0x55a33f674db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 -- 172.21.15.28:0/3309633766 >> 172.21.15.28:0/3309633766 conn(0x55a33f670f60 msgr2=0x55a33f671340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 -- 172.21.15.28:0/3309633766 shutdown_connections 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.425+0000 7fec0510e740 1 -- 172.21.15.28:0/3309633766 wait complete. 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 -- start start 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 Processor -- start 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f67f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6752f0 0x55a33f67c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a33f682780 con 0x55a33f67fbb0 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a33f595f20 con 0x55a33f6749d0 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a33f682900 con 0x55a33f6752f0 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec0510e740 10 client.0 _ll_register_callbacks cb 0x55a33f55f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec012d2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec012d2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43604/0 (socket says 172.21.15.28:43604) 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.426+0000 7fec012d2640 1 -- 172.21.15.28:0/2182500635 learned_addr learned my addr 172.21.15.28:0/2182500635 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f67f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7febfbfff640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6752f0 0x55a33f67c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 -- 172.21.15.28:0/2182500635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6752f0 msgr2=0x55a33f67c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6752f0 0x55a33f67c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 -- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 msgr2=0x55a33f67f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f67f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 -- 172.21.15.28:0/2182500635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33f34c2b0 con 0x55a33f67fbb0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec012d2640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febf4011010 tx=0x7febf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bef31ffd9bbc87d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7febdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4025070 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7febf401f860 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf00aa420 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febf400ee80 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec0510e740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.427+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a33f68e150 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf00aa420 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7febf4012850 con 0x55a33f67fbb0 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 20 client.15462 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f690790 0x55a33f691c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 4 client.15462 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a33f5e8600 con 0x55a33f690790 2024-09-19T09:32:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.428+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f690790 0x55a33f691c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.429+0000 7febdbfff640 10 client.15462 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.429+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f690790 0x55a33f691c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4df143e47d7ce419 server_cookie=e9cbda6565b2d3cb in_seq=0 out_seq=0 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.429+0000 7febdbfff640 1 -- 172.21.15.28:0/2182500635 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a33f5e8600 con 0x55a33f690790 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.429+0000 7febdbfff640 10 client.15462 handle_command_reply: tid=0 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 client.15462 shutdown 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 msgr2=0x55a33f67cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febf4011010 tx=0x7febf4004a40 comp rx=0 tx=0).stop 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 shutdown_connections 2024-09-19T09:32:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f67fbb0 0x55a33f67cdc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f6752f0 0x55a33f67c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f690790 0x55a33f691c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 --2- 172.21.15.28:0/2182500635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f67f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 >> 172.21.15.28:0/2182500635 conn(0x55a33f670f60 msgr2=0x55a33f676c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7febdbfff640 0 client.15462 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 shutdown_connections 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 1 -- 172.21.15.28:0/2182500635 wait complete. 2024-09-19T09:32:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.430+0000 7fec0510e740 20 client.15462 trim_cache size 0 max 16384 2024-09-19T09:32:56.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 Processor -- start 2024-09-19T09:32:56.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 -- start start 2024-09-19T09:32:56.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f6900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f67c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a33f68ff40 con 0x55a33f6752f0 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a33f690610 con 0x55a33f6749d0 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec0510e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a33f690ce0 con 0x55a33f67fbb0 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43612/0 (socket says 172.21.15.28:43612) 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 -- 172.21.15.28:0/2027195338 learned_addr learned my addr 172.21.15.28:0/2027195338 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f6900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 -- 172.21.15.28:0/2027195338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 msgr2=0x55a33f67c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec012d2640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f67c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f67c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 -- 172.21.15.28:0/2027195338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 msgr2=0x55a33f6900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f6900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7febfbfff640 1 -- 172.21.15.28:0/2027195338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33f674800 con 0x55a33f6752f0 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.448+0000 7fec012d2640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f67c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7febfbfff640 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7febe8006010 tx=0x7febe8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfad56e85b99dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7febe67fc640 1 -- 172.21.15.28:0/2027195338 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8035040 con 0x55a33f6752f0 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7febe67fc640 1 -- 172.21.15.28:0/2027195338 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe8031970 con 0x55a33f6752f0 2024-09-19T09:32:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7febe67fc640 1 -- 172.21.15.28:0/2027195338 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8004920 con 0x55a33f6752f0 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 -- 172.21.15.28:0/2027195338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 msgr2=0x55a33f691530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7febe8006010 tx=0x7febe8032f40 comp rx=0 tx=0).stop 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 -- 172.21.15.28:0/2027195338 shutdown_connections 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f67c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6752f0 0x55a33f691530 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 --2- 172.21.15.28:0/2027195338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6749d0 0x55a33f6900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 -- 172.21.15.28:0/2027195338 >> 172.21.15.28:0/2027195338 conn(0x55a33f670f60 msgr2=0x55a33f67bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 -- 172.21.15.28:0/2027195338 shutdown_connections 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.449+0000 7fec0510e740 1 -- 172.21.15.28:0/2027195338 wait complete. 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 -- start start 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 Processor -- start 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6752f0 0x55a33f694a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f695070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a33f691a80 con 0x55a33f6749d0 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a33f6913b0 con 0x55a33f6752f0 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a33f692150 con 0x55a33f67fbb0 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec0510e740 10 client.0 _ll_register_callbacks cb 0x7febf4010d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7febfbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6752f0 0x55a33f694a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec00ad1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43618/0 (socket says 172.21.15.28:43618) 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.450+0000 7fec00ad1640 1 -- 172.21.15.28:0/2170136695 learned_addr learned my addr 172.21.15.28:0/2170136695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec012d2640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f695070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 -- 172.21.15.28:0/2170136695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 msgr2=0x55a33f695070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f695070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 -- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6752f0 msgr2=0x55a33f694a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6752f0 0x55a33f694a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 -- 172.21.15.28:0/2170136695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33f6f8e90 con 0x55a33f6749d0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec00ad1640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febf00a48c0 tx=0x7febf0098420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6279efd4dd223c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7febdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf009c880 con 0x55a33f6749d0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7febf00987f0 con 0x55a33f6749d0 2024-09-19T09:32:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe8031970 con 0x55a33f6749d0 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febf00b5130 con 0x55a33f6749d0 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec0510e740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.451+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a33f6f9f60 con 0x55a33f6749d0 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe8031970 con 0x55a33f6749d0 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7febf00bc0b0 con 0x55a33f6749d0 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 10 client.15468 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 20 client.15468 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f6fe610 0x55a33f6fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 4 client.15468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "status"}) -- 0x55a33f5e2b10 con 0x55a33f6fe610 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.452+0000 7febfbfff640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f6fe610 0x55a33f6fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7febfbfff640 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f6fe610 0x55a33f6fd320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0e1be6125b07442 server_cookie=d7b169a6225c2b5e in_seq=0 out_seq=0 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7febdbfff640 10 client.15468 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7febdbfff640 1 -- 172.21.15.28:0/2170136695 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55a33f5e2b10 con 0x55a33f6fe610 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7febdbfff640 10 client.15468 handle_command_reply: tid=0 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 client.15468 shutdown 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 msgr2=0x55a33f69a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7febf00a48c0 tx=0x7febf0098420 comp rx=0 tx=0).stop 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 shutdown_connections 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55a33f6fe610 0x55a33f6fd320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a33f6752f0 0x55a33f694a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a33f67fbb0 0x55a33f695070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7febdbfff640 0 client.15468 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 --2- 172.21.15.28:0/2170136695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a33f6749d0 0x55a33f69a710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 >> 172.21.15.28:0/2170136695 conn(0x55a33f670f60 msgr2=0x55a33f67c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 shutdown_connections 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 1 -- 172.21.15.28:0/2170136695 wait complete. 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.453+0000 7fec0510e740 20 client.15468 trim_cache size 0 max 16384 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:56.462 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 1, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24482, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 126.308187819, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 27, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 143.37881356 2024-09-19T09:32:56.463 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 msgr2=0x7febec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 0x7febec0a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7febf400bfd0 tx=0x7febf4005c20 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 msgr2=0x7febfc1a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7febf000c920 tx=0x7febf000cde0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 shutdown_connections 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7febec0a32d0 0x7febec0a5790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febfc106f50 0x7febfc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febfc100ca0 0x7febfc1a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 --2- 172.21.15.28:0/4293685656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7febfc1002f0 0x7febfc1a4770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 >> 172.21.15.28:0/4293685656 conn(0x7febfc077210 msgr2=0x7febfc105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:56.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 shutdown_connections 2024-09-19T09:32:56.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:56.454+0000 7fec02d35640 1 -- 172.21.15.28:0/4293685656 wait complete. 2024-09-19T09:32:56.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:56 smithi028 ceph-mon[30972]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T09:32:56.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:56 smithi078 ceph-mon[35619]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T09:32:56.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:56 smithi038 ceph-mon[35271]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T09:32:57.143 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:32:57.401 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.816+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1447323007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 msgr2=0x7f0b0410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.816+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1447323007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b0410bb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009900 tx=0x7f0af402a480 comp rx=0 tx=0).stop 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1447323007 shutdown_connections 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1447323007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0410c170 0x7f0b0410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1447323007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b0410bb50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1447323007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b04102d40 0x7f0b04103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1447323007 >> 172.21.15.28:0/1447323007 conn(0x7f0b040fc7a0 msgr2=0x7f0b040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1447323007 shutdown_connections 2024-09-19T09:32:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1447323007 wait complete. 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 Processor -- start 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.817+0000 7f0b0aa94640 1 -- start start 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b04102d40 0x7f0b04198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b0410c170 0x7f0b0419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04111f70 con 0x7f0b04103660 2024-09-19T09:32:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04111df0 con 0x7f0b0410c170 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b0aa94640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041120f0 con 0x7f0b04102d40 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43630/0 (socket says 172.21.15.28:43630) 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 -- 172.21.15.28:0/1102322020 learned_addr learned my addr 172.21.15.28:0/1102322020 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b09031640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b0410c170 0x7f0b0419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b08830640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b04102d40 0x7f0b04198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 -- 172.21.15.28:0/1102322020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b04102d40 msgr2=0x7f0b04198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b04102d40 0x7f0b04198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 -- 172.21.15.28:0/1102322020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b0410c170 msgr2=0x7f0b0419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b0410c170 0x7f0b0419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 -- 172.21.15.28:0/1102322020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0419d420 con 0x7f0b04103660 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.818+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006010 tx=0x7f0af40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.819+0000 7f0b01ffb640 1 -- 172.21.15.28:0/1102322020 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af403f030 con 0x7f0b04103660 2024-09-19T09:32:57.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.819+0000 7f0b01ffb640 1 -- 172.21.15.28:0/1102322020 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af4003ce0 con 0x7f0b04103660 2024-09-19T09:32:57.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.819+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0419d6b0 con 0x7f0b04103660 2024-09-19T09:32:57.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.819+0000 7f0b01ffb640 1 -- 172.21.15.28:0/1102322020 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4035070 con 0x7f0b04103660 2024-09-19T09:32:57.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.819+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a4ff0 con 0x7f0b04103660 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.821+0000 7f0b01ffb640 1 -- 172.21.15.28:0/1102322020 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0af4042030 con 0x7f0b04103660 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.821+0000 7f0b0ce6d740 1 Processor -- start 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.821+0000 7f0b0ce6d740 1 -- start start 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e741940cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e74194fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e7418a7090 con 0x55e74194ffc0 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e74184ea70 con 0x55e7419408d0 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7418a3050 con 0x55e7419411f0 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b01ffb640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 0x7f0af00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43642/0 (socket says 172.21.15.28:43642) 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b03fff640 1 -- 172.21.15.28:0/2577957361 learned_addr learned my addr 172.21.15.28:0/2577957361 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e741940cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b01ffb640 1 -- 172.21.15.28:0/1102322020 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0af40effa0 con 0x7f0b04103660 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 0x7f0af00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b08830640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e74194fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 -- 172.21.15.28:0/2577957361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 msgr2=0x55e74194fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e74194fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 -- 172.21.15.28:0/2577957361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 msgr2=0x55e7419523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b09031640 1 -- 172.21.15.28:0/2577957361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74185b130 con 0x55e7419408d0 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.822+0000 7f0b03fff640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:32:57.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.823+0000 7f0b09031640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 0x7f0af00a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0afc010c40 tx=0x7f0afc00c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.823+0000 7f0b09031640 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e741940cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005fd0 tx=0x7f0afc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864149e8b7b3abf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.823+0000 7f0afaffd640 1 -- 172.21.15.28:0/2577957361 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc024070 con 0x55e7419408d0 2024-09-19T09:32:57.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.823+0000 7f0afaffd640 1 -- 172.21.15.28:0/2577957361 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc009c40 con 0x55e7419408d0 2024-09-19T09:32:57.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.823+0000 7f0afaffd640 1 -- 172.21.15.28:0/2577957361 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005280 con 0x55e7419408d0 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2577957361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 msgr2=0x55e741940cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e741940cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005fd0 tx=0x7f0afc010740 comp rx=0 tx=0).stop 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2577957361 shutdown_connections 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e74194fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2577957361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e741940cb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2577957361 >> 172.21.15.28:0/2577957361 conn(0x55e74193cf60 msgr2=0x55e74193d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2577957361 shutdown_connections 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2577957361 wait complete. 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 -- start start 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.824+0000 7f0b0ce6d740 1 Processor -- start 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419411f0 0x55e74194cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e74194d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e741952b90 con 0x55e7419408d0 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e741861f20 con 0x55e7419411f0 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e741952d10 con 0x55e74194ffc0 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b0ce6d740 10 client.0 _ll_register_callbacks cb 0x55e74182b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43652/0 (socket says 172.21.15.28:43652) 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 -- 172.21.15.28:0/111248345 learned_addr learned my addr 172.21.15.28:0/111248345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b09031640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e74194d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b03fff640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419411f0 0x55e74194cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 -- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 msgr2=0x55e74194d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e74194d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 -- 172.21.15.28:0/111248345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419411f0 msgr2=0x55e74194cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419411f0 0x55e74194cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 -- 172.21.15.28:0/111248345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7416182b0 con 0x55e7419408d0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0b08830640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00bad0 tx=0x7f0aec00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf088d9fd92f3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.825+0000 7f0ae3fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec003e00 con 0x55e7419408d0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0aec005020 con 0x55e7419408d0 2024-09-19T09:32:57.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0afc009c40 con 0x55e7419408d0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0aec005220 con 0x55e7419408d0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e74195e600 con 0x55e7419408d0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc009c40 con 0x55e7419408d0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0aec01b630 con 0x55e7419408d0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 20 client.15480 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e741960c10 0x55e741962080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 4 client.15480 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.826+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7418b4600 con 0x55e741960c10 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.827+0000 7f0b03fff640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e741960c10 0x55e741962080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.827+0000 7f0b03fff640 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e741960c10 0x55e741962080 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b300cceb2a9c4f server_cookie=d21c02ea919a3f4b in_seq=0 out_seq=0 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.827+0000 7f0ae3fff640 10 client.15480 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0ae3fff640 1 -- 172.21.15.28:0/111248345 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7418b4600 con 0x55e741960c10 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0ae3fff640 10 client.15480 handle_command_reply: tid=0 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 client.15480 shutdown 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 msgr2=0x55e74194f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00bad0 tx=0x7f0aec00bf90 comp rx=0 tx=0).stop 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 shutdown_connections 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419411f0 0x55e74194cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e741960c10 0x55e741962080 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e74194d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/111248345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419408d0 0x55e74194f910 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 >> 172.21.15.28:0/111248345 conn(0x55e74193cf60 msgr2=0x55e741942c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.828+0000 7f0ae3fff640 0 client.15480 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:57.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.829+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 shutdown_connections 2024-09-19T09:32:57.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.829+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/111248345 wait complete. 2024-09-19T09:32:57.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.829+0000 7f0b0ce6d740 20 client.15480 trim_cache size 0 max 16384 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.846+0000 7f0b0ce6d740 1 Processor -- start 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.846+0000 7f0b0ce6d740 1 -- start start 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.846+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e74194d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e741941a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e7419603c0 con 0x55e7419411f0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e741960a90 con 0x55e7419408d0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e741961160 con 0x55e74194ffc0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43656/0 (socket says 172.21.15.28:43656) 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 -- 172.21.15.28:0/1598844589 learned_addr learned my addr 172.21.15.28:0/1598844589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b08830640 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e74194d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 -- 172.21.15.28:0/1598844589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 msgr2=0x55e741941a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e741941a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 -- 172.21.15.28:0/1598844589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 msgr2=0x55e74194d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e74194d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 -- 172.21.15.28:0/1598844589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e741952d00 con 0x55e7419411f0 2024-09-19T09:32:57.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b08830640 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e74194d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:32:57.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0b03fff640 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0af40b8fd0 tx=0x7f0af40e7260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990fe6eb61c5ab1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0afa7fc640 1 -- 172.21.15.28:0/1598844589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40da070 con 0x55e7419411f0 2024-09-19T09:32:57.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0afa7fc640 1 -- 172.21.15.28:0/1598844589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af40bc600 con 0x55e7419411f0 2024-09-19T09:32:57.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.847+0000 7f0afa7fc640 1 -- 172.21.15.28:0/1598844589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4049040 con 0x55e7419411f0 2024-09-19T09:32:57.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/1598844589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 msgr2=0x55e7419612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0af40b8fd0 tx=0x7f0af40e7260 comp rx=0 tx=0).stop 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/1598844589 shutdown_connections 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e74194ffc0 0x55e741941a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e7419411f0 0x55e7419612e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/1598844589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e74194d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/1598844589 >> 172.21.15.28:0/1598844589 conn(0x55e74193cf60 msgr2=0x55e74193d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/1598844589 shutdown_connections 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/1598844589 wait complete. 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 -- start start 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.848+0000 7f0b0ce6d740 1 Processor -- start 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e7419680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e741968610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e741961f00 con 0x55e74194ffc0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e741961830 con 0x55e7419408d0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7419625d0 con 0x55e7419411f0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b0ce6d740 10 client.0 _ll_register_callbacks cb 0x7f0aec004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43666/0 (socket says 172.21.15.28:43666) 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 -- 172.21.15.28:0/2695257898 learned_addr learned my addr 172.21.15.28:0/2695257898 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b08830640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e7419680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b03fff640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e741968610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 -- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 msgr2=0x55e741968610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e741968610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 -- 172.21.15.28:0/2695257898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 msgr2=0x55e7419680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:32:57.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e7419680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b09031640 1 -- 172.21.15.28:0/2695257898 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74193c810 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.849+0000 7f0b03fff640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e741968610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0b09031640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009b10 tx=0x7f0afc009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4532b529735de813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc024040 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0afc004a00 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af40bc600 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0afc009c40 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0b0ce6d740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e741967490 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af40bc600 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0afc030070 con 0x55e74194ffc0 2024-09-19T09:32:57.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.850+0000 7f0b0ce6d740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 10 client.15486 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 20 client.15486 populate_metadata read hostname 'smithi028' 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e7419cf3c0 0x55e7419ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 4 client.15486 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "status"}) -- 0x55e7418aeb10 con 0x55e7419cf3c0 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b08830640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e7419cf3c0 0x55e7419ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b08830640 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e7419cf3c0 0x55e7419ce0d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac9295b2e523144 server_cookie=f3ae68f73024887d in_seq=0 out_seq=0 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0ae3fff640 10 client.15486 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0ae3fff640 1 -- 172.21.15.28:0/2695257898 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e7418aeb10 con 0x55e7419cf3c0 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0ae3fff640 10 client.15486 handle_command_reply: tid=0 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.851+0000 7f0b0ce6d740 1 client.15486 shutdown 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 msgr2=0x55e7419636b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009b10 tx=0x7f0afc009670 comp rx=0 tx=0).stop 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 shutdown_connections 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e7419cf3c0 0x55e7419ce0d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e74194ffc0 0x55e7419636b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7419411f0 0x55e741968610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 --2- 172.21.15.28:0/2695257898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7419408d0 0x55e7419680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0ae3fff640 0 client.15486 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 >> 172.21.15.28:0/2695257898 conn(0x55e74193cf60 msgr2=0x55e74193b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 shutdown_connections 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 1 -- 172.21.15.28:0/2695257898 wait complete. 2024-09-19T09:32:57.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0ce6d740 20 client.15486 trim_cache size 0 max 16384 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "9a2e9aa4-7668-11ef-bceb-c7b262605968", 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 24499, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 134.299557209, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 27, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 82, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T09:32:57.860 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 153.13838905399999 2024-09-19T09:32:57.861 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:32:57.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 msgr2=0x7f0af00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 0x7f0af00a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0afc010c40 tx=0x7f0afc00c000 comp rx=0 tx=0).stop 2024-09-19T09:32:57.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 msgr2=0x7f0b041989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:32:57.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.852+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006010 tx=0x7f0af40042f0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 shutdown_connections 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0af00a32d0 0x7f0af00a5790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b0410c170 0x7f0b0419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b04103660 0x7f0b041989b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 --2- 172.21.15.28:0/1102322020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b04102d40 0x7f0b04198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 >> 172.21.15.28:0/1102322020 conn(0x7f0b040fc7a0 msgr2=0x7f0b040fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 shutdown_connections 2024-09-19T09:32:57.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:32:57.853+0000 7f0b0aa94640 1 -- 172.21.15.28:0/1102322020 wait complete. 2024-09-19T09:32:58.637 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T09:32:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:32:58 smithi028 ceph-mon[30972]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-09-19T09:32:58.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:32:58 smithi078 ceph-mon[35619]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-09-19T09:32:58.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:32:58 smithi038 ceph-mon[35271]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 14 op/s 2024-09-19T09:33:00.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:00 smithi028 ceph-mon[30972]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-09-19T09:33:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:00 smithi078 ceph-mon[35619]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-09-19T09:33:00.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:00 smithi038 ceph-mon[35271]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-09-19T09:33:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4092282507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 msgr2=0x7fd7a4108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4092282507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4108eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009ca0 tx=0x7fd79c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4092282507 shutdown_connections 2024-09-19T09:33:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4092282507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4108eb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4092282507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 0x7fd7a41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4092282507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4092282507 >> 172.21.15.28:0/4092282507 conn(0x7fd7a40fc1f0 msgr2=0x7fd7a40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4092282507 shutdown_connections 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.326+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4092282507 wait complete. 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 Processor -- start 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 -- start start 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a410b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 0x7fd7a4109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a410d800 con 0x7fd7a4102620 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a410d680 con 0x7fd7a4100a70 2024-09-19T09:33:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a9f11640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a410d980 con 0x7fd7a4101c60 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43686/0 (socket says 172.21.15.28:43686) 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 -- 172.21.15.28:0/4087036712 learned_addr learned my addr 172.21.15.28:0/4087036712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd79bfff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 0x7fd7a4109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a410b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 -- 172.21.15.28:0/4087036712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 msgr2=0x7fd7a4109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 0x7fd7a4109a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 -- 172.21.15.28:0/4087036712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 msgr2=0x7fd7a410b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a410b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a3fff640 1 -- 172.21.15.28:0/4087036712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a410a800 con 0x7fd7a4102620 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.327+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a3fff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd79c031ee0 tx=0x7fd79c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a17fa640 1 -- 172.21.15.28:0/4087036712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c003c30 con 0x7fd7a4102620 2024-09-19T09:33:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a17fa640 1 -- 172.21.15.28:0/4087036712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c003dd0 con 0x7fd7a4102620 2024-09-19T09:33:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a17fa640 1 -- 172.21.15.28:0/4087036712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c02fc10 con 0x7fd7a4102620 2024-09-19T09:33:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a41a1fb0 con 0x7fd7a4102620 2024-09-19T09:33:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.328+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41a2360 con 0x7fd7a4102620 2024-09-19T09:33:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.330+0000 7fd7a17fa640 1 -- 172.21.15.28:0/4087036712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd79c0046c0 con 0x7fd7a4102620 2024-09-19T09:33:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 Processor -- start 2024-09-19T09:33:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 -- start start 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861efeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 0x555861f06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555861e622f0 con 0x555861eff2e0 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555861e0a270 con 0x555861efe9a0 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7acca5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555861e5e2b0 con 0x555861f04370 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43702/0 (socket says 172.21.15.28:43702) 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 -- 172.21.15.28:0/3378291067 learned_addr learned my addr 172.21.15.28:0/3378291067 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd79bfff640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861efeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd79bfff640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 0x555861f06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a17fa640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 0x7fd7900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 -- 172.21.15.28:0/3378291067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 msgr2=0x555861f06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 0x555861f06830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 -- 172.21.15.28:0/3378291067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 msgr2=0x555861efeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861efeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a37fe640 1 -- 172.21.15.28:0/3378291067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555861e09340 con 0x555861eff2e0 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd7a3fff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 0x7fd7900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.331+0000 7fd79bfff640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 0x555861f06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.332+0000 7fd7a17fa640 1 -- 172.21.15.28:0/4087036712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd79c0f0c10 con 0x7fd7a4102620 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.332+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd79400bd40 tx=0x7fd79400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eb387a0be5154c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.332+0000 7fd799ffb640 1 -- 172.21.15.28:0/3378291067 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794011640 con 0x555861eff2e0 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.332+0000 7fd799ffb640 1 -- 172.21.15.28:0/3378291067 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79400be30 con 0x555861eff2e0 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.332+0000 7fd799ffb640 1 -- 172.21.15.28:0/3378291067 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794010bc0 con 0x555861eff2e0 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- 172.21.15.28:0/3378291067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 msgr2=0x555861f03da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7a3fff640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 0x7fd7900a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0bd1a0 tx=0x7fd79c0a7b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd79400bd40 tx=0x7fd79400f730 comp rx=0 tx=0).stop 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- 172.21.15.28:0/3378291067 shutdown_connections 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861f04370 0x555861f06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861eff2e0 0x555861f03da0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 --2- 172.21.15.28:0/3378291067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861efeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- 172.21.15.28:0/3378291067 >> 172.21.15.28:0/3378291067 conn(0x555861efa010 msgr2=0x555861efa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- 172.21.15.28:0/3378291067 shutdown_connections 2024-09-19T09:33:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- 172.21.15.28:0/3378291067 wait complete. 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 -- start start 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.333+0000 7fd7acca5740 1 Processor -- start 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861f08bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861eff2e0 0x555861f09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555861f07340 con 0x555861f04370 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555861deb290 con 0x555861efe9a0 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555861f074c0 con 0x555861eff2e0 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7acca5740 10 client.0 _ll_register_callbacks cb 0x555861e1c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43714/0 (socket says 172.21.15.28:43714) 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 -- 172.21.15.28:0/1750323201 learned_addr learned my addr 172.21.15.28:0/1750323201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861f08bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861eff2e0 0x555861f09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 -- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861eff2e0 msgr2=0x555861f09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861eff2e0 0x555861f09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 -- 172.21.15.28:0/1750323201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 msgr2=0x555861f08bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861f08bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 -- 172.21.15.28:0/1750323201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555861b75210 con 0x555861f04370 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd78c012c70 tx=0x7fd78c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528af05af5296692 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.334+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c01b040 con 0x555861f04370 2024-09-19T09:33:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd78c004030 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79400be30 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd78c028020 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555861c41620 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79400be30 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd78c0047a0 con 0x555861f04370 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 20 client.15498 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f148c0 0x555861f16d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 4 client.15498 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555861e6fa20 con 0x555861f148c0 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.335+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f148c0 0x555861f16d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.336+0000 7fd782ffd640 10 client.15498 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.336+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f148c0 0x555861f16d10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa7dab179706594 server_cookie=72009f2d45a408d6 in_seq=0 out_seq=0 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.336+0000 7fd782ffd640 1 -- 172.21.15.28:0/1750323201 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555861e6fa20 con 0x555861f148c0 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.336+0000 7fd782ffd640 10 client.15498 handle_command_reply: tid=0 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.336+0000 7fd7acca5740 1 client.15498 shutdown 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 msgr2=0x555861f10170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd78c012c70 tx=0x7fd78c004730 comp rx=0 tx=0).stop 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 shutdown_connections 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f148c0 0x555861f16d10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f10170 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861eff2e0 0x555861f09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1750323201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861efe9a0 0x555861f08bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 >> 172.21.15.28:0/1750323201 conn(0x555861efa010 msgr2=0x555861efb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd782ffd640 0 client.15498 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 shutdown_connections 2024-09-19T09:33:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 1 -- 172.21.15.28:0/1750323201 wait complete. 2024-09-19T09:33:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.337+0000 7fd7acca5740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.351+0000 7fd7acca5740 1 Processor -- start 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.351+0000 7fd7acca5740 1 -- start start 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861efaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41ab200 con 0x555861f04370 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41acd40 con 0x555861eff2e0 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7acca5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41ab8d0 con 0x555861efe9a0 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43722/0 (socket says 172.21.15.28:43722) 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 -- 172.21.15.28:0/1987256385 learned_addr learned my addr 172.21.15.28:0/1987256385 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861efaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 -- 172.21.15.28:0/1987256385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 msgr2=0x555861efaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861efaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 -- 172.21.15.28:0/1987256385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 msgr2=0x555861f100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 -- 172.21.15.28:0/1987256385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555861f13230 con 0x555861f04370 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001eb0 tx=0x7fd78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b7c6d21bde4e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7997fa640 1 -- 172.21.15.28:0/1987256385 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0125b0 con 0x555861f04370 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7997fa640 1 -- 172.21.15.28:0/1987256385 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c0047a0 con 0x555861f04370 2024-09-19T09:33:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.352+0000 7fd7997fa640 1 -- 172.21.15.28:0/1987256385 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c02d6c0 con 0x555861f04370 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- 172.21.15.28:0/1987256385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 msgr2=0x555861efb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001eb0 tx=0x7fd78c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- 172.21.15.28:0/1987256385 shutdown_connections 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861efb400 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1987256385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861efaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- 172.21.15.28:0/1987256385 >> 172.21.15.28:0/1987256385 conn(0x555861efa010 msgr2=0x555861f08bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- 172.21.15.28:0/1987256385 shutdown_connections 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- 172.21.15.28:0/1987256385 wait complete. 2024-09-19T09:33:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.353+0000 7fd7acca5740 1 -- start start 2024-09-19T09:33:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 Processor -- start 2024-09-19T09:33:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861f0d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f0dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41ac670 con 0x555861f04370 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41abfa0 con 0x555861eff2e0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41a91e0 con 0x555861efe9a0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7acca5740 10 client.0 _ll_register_callbacks cb 0x7fd78c00bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43728/0 (socket says 172.21.15.28:43728) 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 -- 172.21.15.28:0/1733884399 learned_addr learned my addr 172.21.15.28:0/1733884399 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861f0d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f0dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 -- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 msgr2=0x555861f0d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861f0d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 -- 172.21.15.28:0/1733884399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 msgr2=0x555861f0dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f0dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 -- 172.21.15.28:0/1733884399 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555861f0f340 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd79bfff640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcf0 tx=0x7fd78c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14016f9084ad1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.354+0000 7fd782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c004fe0 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd78c005180 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c0047a0 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd78c004950 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555861f7f650 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c0047a0 con 0x555861f04370 2024-09-19T09:33:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd78c03c450 con 0x555861f04370 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 20 client.15504 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f7ff20 0x555861f82340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 4 client.15504 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555861e69d70 con 0x555861f7ff20 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.355+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f7ff20 0x555861f82340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.356+0000 7fd7a37fe640 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f7ff20 0x555861f82340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfb24180e13141e server_cookie=6eef3de6fdd6d3cf in_seq=0 out_seq=0 2024-09-19T09:33:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.356+0000 7fd782ffd640 10 client.15504 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd782ffd640 1 -- 172.21.15.28:0/1733884399 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555861e69d70 con 0x555861f7ff20 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd782ffd640 10 client.15504 handle_command_reply: tid=0 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 client.15504 shutdown 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 msgr2=0x555861f0ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcf0 tx=0x7fd78c004730 comp rx=0 tx=0).stop 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 shutdown_connections 2024-09-19T09:33:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555861eff2e0 0x555861f0dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555861f7ff20 0x555861f82340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555861f04370 0x555861f0ed30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 --2- 172.21.15.28:0/1733884399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555861efe9a0 0x555861f0d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd782ffd640 0 client.15504 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 >> 172.21.15.28:0/1733884399 conn(0x555861efa010 msgr2=0x555861ef89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 shutdown_connections 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 1 -- 172.21.15.28:0/1733884399 wait complete. 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.364+0000 7fd7acca5740 20 client.15504 trim_cache size 0 max 16384 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:33:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:33:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:33:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:33:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 msgr2=0x7fd7900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 0x7fd7900a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0bd1a0 tx=0x7fd79c0a7b50 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 msgr2=0x7fd7a4109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd79c031ee0 tx=0x7fd79c031f10 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 shutdown_connections 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd7900a32d0 0x7fd7900a5790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7a4102620 0x7fd7a4109f70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7a4101c60 0x7fd7a4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 --2- 172.21.15.28:0/4087036712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4100a70 0x7fd7a410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 >> 172.21.15.28:0/4087036712 conn(0x7fd7a40fc1f0 msgr2=0x7fd7a4105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 shutdown_connections 2024-09-19T09:33:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:02.365+0000 7fd7a9f11640 1 -- 172.21.15.28:0/4087036712 wait complete. 2024-09-19T09:33:02.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:02 smithi028 ceph-mon[30972]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-19T09:33:02.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:02 smithi078 ceph-mon[35619]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-19T09:33:02.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:02 smithi038 ceph-mon[35271]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-19T09:33:04.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:04 smithi028 ceph-mon[30972]: pgmap v350: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-19T09:33:04.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:04 smithi078 ceph-mon[35619]: pgmap v350: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-19T09:33:04.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:04 smithi038 ceph-mon[35271]: pgmap v350: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-19T09:33:06.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:06 smithi028 ceph-mon[30972]: pgmap v351: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T09:33:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:06 smithi078 ceph-mon[35619]: pgmap v351: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T09:33:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:06 smithi038 ceph-mon[35271]: pgmap v351: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T09:33:08.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:08 smithi028 ceph-mon[30972]: pgmap v352: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:08.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:08 smithi078 ceph-mon[35619]: pgmap v352: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:08.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:08 smithi038 ceph-mon[35271]: pgmap v352: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:09.960 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:33:10.237 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:10.620 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:10 smithi028 ceph-mon[30972]: pgmap v353: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:33:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 -- 172.21.15.28:0/2197190423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 msgr2=0x7f5cac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 --2- 172.21.15.28:0/2197190423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac10da40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009be0 tx=0x7f5ca402a750 comp rx=0 tx=0).stop 2024-09-19T09:33:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 -- 172.21.15.28:0/2197190423 shutdown_connections 2024-09-19T09:33:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 --2- 172.21.15.28:0/2197190423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac102570 0x7f5cac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 --2- 172.21.15.28:0/2197190423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac10da40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 --2- 172.21.15.28:0/2197190423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac100e20 0x7f5cac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 -- 172.21.15.28:0/2197190423 >> 172.21.15.28:0/2197190423 conn(0x7f5cac0fc7d0 msgr2=0x7f5cac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 -- 172.21.15.28:0/2197190423 shutdown_connections 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.707+0000 7f5cb100d640 1 -- 172.21.15.28:0/2197190423 wait complete. 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 Processor -- start 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 -- start start 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac100e20 0x7f5cac0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac102570 0x7f5cac0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac110f30 con 0x7f5cac102ef0 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac110db0 con 0x7f5cac100e20 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cb100d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1110b0 con 0x7f5cac102570 2024-09-19T09:33:10.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cab59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cab59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59118/0 (socket says 172.21.15.28:59118) 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.708+0000 7f5cab59e640 1 -- 172.21.15.28:0/1567090269 learned_addr learned my addr 172.21.15.28:0/1567090269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5caad9d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac100e20 0x7f5cac0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac102570 0x7f5cac0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 -- 172.21.15.28:0/1567090269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac102570 msgr2=0x7f5cac0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac102570 0x7f5cac0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 -- 172.21.15.28:0/1567090269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac100e20 msgr2=0x7f5cac0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac100e20 0x7f5cac0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 -- 172.21.15.28:0/1567090269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac1a6930 con 0x7f5cac102ef0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cab59e640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009d00 tx=0x7f5ca4030cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5ca8d99640 1 -- 172.21.15.28:0/1567090269 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4041020 con 0x7f5cac102ef0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac1a6bc0 con 0x7f5cac102ef0 2024-09-19T09:33:10.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5ca8d99640 1 -- 172.21.15.28:0/1567090269 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca4037040 con 0x7f5cac102ef0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5ca8d99640 1 -- 172.21.15.28:0/1567090269 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4036070 con 0x7f5cac102ef0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.709+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a70f0 con 0x7f5cac102ef0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.712+0000 7f5ca8d99640 1 -- 172.21.15.28:0/1567090269 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f5ca4044020 con 0x7f5cac102ef0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.712+0000 7f5cb33e6740 1 Processor -- start 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 -- start start 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61caf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556f61c152e0 con 0x556f61cafdc0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556f61bbcce0 con 0x556f61cba680 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556f61c112a0 con 0x556f61caf4a0 2024-09-19T09:33:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59124/0 (socket says 172.21.15.28:59124) 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 -- 172.21.15.28:0/545556212 learned_addr learned my addr 172.21.15.28:0/545556212 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61caf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5ca8d99640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 0x7f5c980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 -- 172.21.15.28:0/545556212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 msgr2=0x556f61caf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61caf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 -- 172.21.15.28:0/545556212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 msgr2=0x556f61cbca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.713+0000 7f5caad9d640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5caad9d640 1 -- 172.21.15.28:0/545556212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f61ba1440 con 0x556f61cafdc0 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61caf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5ca8d99640 1 -- 172.21.15.28:0/1567090269 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5ca40f0560 con 0x7f5cac102ef0 2024-09-19T09:33:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5cab59e640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 0x7f5c980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5caad9d640 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8960c2403af7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5ca17fa640 1 -- 172.21.15.28:0/545556212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c01e070 con 0x556f61cafdc0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5ca17fa640 1 -- 172.21.15.28:0/545556212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c019b10 con 0x556f61cafdc0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.714+0000 7f5ca17fa640 1 -- 172.21.15.28:0/545556212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ebf0 con 0x556f61cafdc0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 -- 172.21.15.28:0/545556212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 msgr2=0x556f61cba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cab59e640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 0x7f5c980a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ca408f8a0 tx=0x7f5ca40a7040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011810 comp rx=0 tx=0).stop 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 -- 172.21.15.28:0/545556212 shutdown_connections 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cba140 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/545556212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61caf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 -- 172.21.15.28:0/545556212 >> 172.21.15.28:0/545556212 conn(0x556f61caba30 msgr2=0x556f61cabe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 -- 172.21.15.28:0/545556212 shutdown_connections 2024-09-19T09:33:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.715+0000 7f5cb33e6740 1 -- 172.21.15.28:0/545556212 wait complete. 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 -- start start 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 Processor -- start 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 0x556f61cb9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61cba680 0x556f61cb7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556f61cbd250 con 0x556f61cafdc0 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 0x556f61cb9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59138/0 (socket says 172.21.15.28:59138) 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5caad9d640 1 -- 172.21.15.28:0/3420115425 learned_addr learned my addr 172.21.15.28:0/3420115425 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556f61bd0050 con 0x556f61caf4a0 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.716+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556f61cbd3d0 con 0x556f61cba680 2024-09-19T09:33:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5cb33e6740 10 client.0 _ll_register_callbacks cb 0x556f61c10160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61cba680 0x556f61cb7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 -- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61cba680 msgr2=0x556f61cb7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61cba680 0x556f61cb7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 -- 172.21.15.28:0/3420115425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 msgr2=0x556f61cb9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 0x556f61cb9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 -- 172.21.15.28:0/3420115425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f619863c0 con 0x556f61cafdc0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 0x556f61cb9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5caad9d640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c011d10 tx=0x7f5c9c019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3289e70d2d1927e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c01e070 con 0x556f61cafdc0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5c9c00eaf0 con 0x556f61cafdc0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c019b10 con 0x556f61cafdc0 2024-09-19T09:33:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0098f0 con 0x556f61cafdc0 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.717+0000 7f5cb33e6740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f61cc8bb0 con 0x556f61cafdc0 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c019b10 con 0x556f61cafdc0 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5c9c005210 con 0x556f61cafdc0 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 20 client.15516 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61cca010 0x556f61ccc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 4 client.15516 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f61c22850 con 0x556f61cca010 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.718+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61cca010 0x556f61ccc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.719+0000 7f5c927fc640 10 client.15516 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.719+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61cca010 0x556f61ccc400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d9b4d4b67ac3e8 server_cookie=2c217b33e3b9e999 in_seq=0 out_seq=0 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5c927fc640 1 -- 172.21.15.28:0/3420115425 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f61c22850 con 0x556f61cca010 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5c927fc640 10 client.15516 handle_command_reply: tid=0 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 client.15516 shutdown 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 msgr2=0x556f61cb7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c011d10 tx=0x7f5c9c019660 comp rx=0 tx=0).stop 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 shutdown_connections 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61cca010 0x556f61ccc400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5c927fc640 0 client.15516 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61cba680 0x556f61cb7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cb7210 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/3420115425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61caf4a0 0x556f61cb9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 >> 172.21.15.28:0/3420115425 conn(0x556f61caba30 msgr2=0x556f61cb1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.720+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 shutdown_connections 2024-09-19T09:33:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.721+0000 7f5cb33e6740 1 -- 172.21.15.28:0/3420115425 wait complete. 2024-09-19T09:33:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.721+0000 7f5cb33e6740 20 client.15516 trim_cache size 0 max 16384 2024-09-19T09:33:10.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 Processor -- start 2024-09-19T09:33:10.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 -- start start 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cb8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 0x556f61cd1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1af600 con 0x556f61cba680 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1b1140 con 0x556f61cafdc0 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1afcd0 con 0x556f61caf4a0 2024-09-19T09:33:10.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.738+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59144/0 (socket says 172.21.15.28:59144) 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 -- 172.21.15.28:0/1302417526 learned_addr learned my addr 172.21.15.28:0/1302417526 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5caad9d640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 0x556f61cd1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cb8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 -- 172.21.15.28:0/1302417526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 msgr2=0x556f61cb8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cb8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 -- 172.21.15.28:0/1302417526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 msgr2=0x556f61cd1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 0x556f61cd1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 -- 172.21.15.28:0/1302417526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f61cb9920 con 0x556f61cba680 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5caad9d640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 0x556f61cd1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e75d4e39a02a3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca0ff9640 1 -- 172.21.15.28:0/1302417526 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9401b040 con 0x556f61cba680 2024-09-19T09:33:10.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca0ff9640 1 -- 172.21.15.28:0/1302417526 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c94005500 con 0x556f61cba680 2024-09-19T09:33:10.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.739+0000 7f5ca0ff9640 1 -- 172.21.15.28:0/1302417526 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94004dd0 con 0x556f61cba680 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 -- 172.21.15.28:0/1302417526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 msgr2=0x556f61cd2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c940046c0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 -- 172.21.15.28:0/1302417526 shutdown_connections 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cafdc0 0x556f61cd1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cba680 0x556f61cd2170 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/1302417526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cb8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 -- 172.21.15.28:0/1302417526 >> 172.21.15.28:0/1302417526 conn(0x556f61caba30 msgr2=0x556f61cb61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 -- 172.21.15.28:0/1302417526 shutdown_connections 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.740+0000 7f5cb33e6740 1 -- 172.21.15.28:0/1302417526 wait complete. 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 -- start start 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 Processor -- start 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cc2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1b0a70 con 0x556f61cafdc0 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1b03a0 con 0x556f61cba680 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1ad5e0 con 0x556f61caf4a0 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5cb33e6740 10 client.0 _ll_register_callbacks cb 0x7f5c9c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5caad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59152/0 (socket says 172.21.15.28:59152) 2024-09-19T09:33:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5caad9d640 1 -- 172.21.15.28:0/2617090232 learned_addr learned my addr 172.21.15.28:0/2617090232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cc2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.741+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 -- 172.21.15.28:0/2617090232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 msgr2=0x556f61cbffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 -- 172.21.15.28:0/2617090232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 msgr2=0x556f61cc2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 -- 172.21.15.28:0/2617090232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f61cc07d0 con 0x556f61caf4a0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5caad9d640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5ca3fff640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cc2a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c94004fd0 tx=0x7f5c94016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234ff2561e1eb661 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94021c50 con 0x556f61caf4a0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c94005500 con 0x556f61caf4a0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5c94031030 con 0x556f61caf4a0 2024-09-19T09:33:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c94014c70 con 0x556f61caf4a0 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5cb33e6740 10 client.25135 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.742+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f61d34890 con 0x556f61caf4a0 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c94005500 con 0x556f61caf4a0 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5c94021040 con 0x556f61caf4a0 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 10 client.25135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 10 client.25135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 20 client.25135 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61d34f00 0x556f61d372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 4 client.25135 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f61c1cd60 con 0x556f61d34f00 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5caad9d640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61d34f00 0x556f61d372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5c927fc640 10 client.25135 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.743+0000 7f5caad9d640 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61d34f00 0x556f61d372f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3421b65bd210835e server_cookie=18bb61738908a0e8 in_seq=0 out_seq=0 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5c927fc640 1 -- 172.21.15.28:0/2617090232 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f61c1cd60 con 0x556f61d34f00 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5c927fc640 10 client.25135 handle_command_reply: tid=0 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 client.25135 shutdown 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 msgr2=0x556f61cc2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cc2a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c94004fd0 tx=0x7f5c94016040 comp rx=0 tx=0).stop 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 shutdown_connections 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556f61cba680 0x556f61cbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556f61d34f00 0x556f61d372f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556f61cafdc0 0x556f61cc2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 --2- 172.21.15.28:0/2617090232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f61caf4a0 0x556f61cc2a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5c927fc640 0 client.25135 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 >> 172.21.15.28:0/2617090232 conn(0x556f61caba30 msgr2=0x556f61cb59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 shutdown_connections 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 1 -- 172.21.15.28:0/2617090232 wait complete. 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.744+0000 7f5cb33e6740 20 client.25135 trim_cache size 0 max 16384 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 msgr2=0x7f5c980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 0x7f5c980a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ca408f8a0 tx=0x7f5ca40a7040 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 msgr2=0x7f5cac0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009d00 tx=0x7f5ca4030cb0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 shutdown_connections 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c980a32d0 0x7f5c980a5790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cac102ef0 0x7f5cac0fd700 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cac102570 0x7f5cac0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 --2- 172.21.15.28:0/1567090269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cac100e20 0x7f5cac0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 >> 172.21.15.28:0/1567090269 conn(0x7f5cac0fc7d0 msgr2=0x7f5cac06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 shutdown_connections 2024-09-19T09:33:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:10.745+0000 7f5cb100d640 1 -- 172.21.15.28:0/1567090269 wait complete. 2024-09-19T09:33:10.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:10 smithi078 ceph-mon[35619]: pgmap v353: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:33:10.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:10 smithi038 ceph-mon[35271]: pgmap v353: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:33:11.513 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:33:11.514 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:33:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 -- 172.21.15.28:0/2965459818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c101de0 msgr2=0x7fd73c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 --2- 172.21.15.28:0/2965459818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c101de0 0x7fd73c1021c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002a410 comp rx=0 tx=0).stop 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 -- 172.21.15.28:0/2965459818 shutdown_connections 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 --2- 172.21.15.28:0/2965459818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c102790 0x7fd73c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 --2- 172.21.15.28:0/2965459818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c106f10 0x7fd73c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 --2- 172.21.15.28:0/2965459818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c101de0 0x7fd73c1021c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 -- 172.21.15.28:0/2965459818 >> 172.21.15.28:0/2965459818 conn(0x7fd73c0fd8c0 msgr2=0x7fd73c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 -- 172.21.15.28:0/2965459818 shutdown_connections 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.232+0000 7fd742f33640 1 -- 172.21.15.28:0/2965459818 wait complete. 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 Processor -- start 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 -- start start 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 0x7fd73c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 0x7fd73c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111f70 con 0x7fd73c102790 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111df0 con 0x7fd73c106f10 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd742f33640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1120f0 con 0x7fd73c101de0 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59178/0 (socket says 172.21.15.28:59178) 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 -- 172.21.15.28:0/2938570088 learned_addr learned my addr 172.21.15.28:0/2938570088 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd740ccf640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 0x7fd73c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd7414d0640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 0x7fd73c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 -- 172.21.15.28:0/2938570088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 msgr2=0x7fd73c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 0x7fd73c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 -- 172.21.15.28:0/2938570088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 msgr2=0x7fd73c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 0x7fd73c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.233+0000 7fd73bfff640 1 -- 172.21.15.28:0/2938570088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c19f660 con 0x7fd73c102790 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd740ccf640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 0x7fd73c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd7414d0640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 0x7fd73c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd73bfff640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9f0 tx=0x7fd72800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72800cd80 con 0x7fd73c102790 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728010910 con 0x7fd73c102790 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c19f950 con 0x7fd73c102790 2024-09-19T09:33:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728018990 con 0x7fd73c102790 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.234+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1a7230 con 0x7fd73c102790 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.235+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c1021c0 con 0x7fd73c102790 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.236+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd728010ac0 con 0x7fd73c102790 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.238+0000 7fd739ffb640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 0x7fd72c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.238+0000 7fd740ccf640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 0x7fd72c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.238+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fd7280cfe40 con 0x7fd73c102790 2024-09-19T09:33:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.239+0000 7fd740ccf640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 0x7fd72c0a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd730005ab0 tx=0x7fd730005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:12.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.242+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd72809ca30 con 0x7fd73c102790 2024-09-19T09:33:12.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.459+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd73c1a7400 con 0x7fd73c102790 2024-09-19T09:33:12.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.460+0000 7fd739ffb640 1 -- 172.21.15.28:0/2938570088 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7fd72809c330 con 0x7fd73c102790 2024-09-19T09:33:12.461 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:33:12.461 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:33:12.461 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:33:12.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.462+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 msgr2=0x7fd72c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:12.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.462+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 0x7fd72c0a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd730005ab0 tx=0x7fd730005a20 comp rx=0 tx=0).stop 2024-09-19T09:33:12.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.462+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 msgr2=0x7fd73c19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.462+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9f0 tx=0x7fd72800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 shutdown_connections 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd72c0a3350 0x7fd72c0a5810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd73c106f10 0x7fd73c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c102790 0x7fd73c19abf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 --2- 172.21.15.28:0/2938570088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd73c101de0 0x7fd73c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 >> 172.21.15.28:0/2938570088 conn(0x7fd73c0fd8c0 msgr2=0x7fd73c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 shutdown_connections 2024-09-19T09:33:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:12.463+0000 7fd742f33640 1 -- 172.21.15.28:0/2938570088 wait complete. 2024-09-19T09:33:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:12 smithi028 ceph-mon[30972]: pgmap v354: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:33:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:12 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2938570088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:12 smithi078 ceph-mon[35619]: pgmap v354: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:33:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:12 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2938570088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:12.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:12 smithi038 ceph-mon[35271]: pgmap v354: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:33:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:12 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2938570088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:13.104 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:33:13.362 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:13.639 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T09:33:13.639 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 -- 172.21.15.28:0/1369546472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0100e20 msgr2=0x7f91a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 --2- 172.21.15.28:0/1369546472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0100e20 0x7f91a0101200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f919c009900 tx=0x7f919c02a480 comp rx=0 tx=0).stop 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 -- 172.21.15.28:0/1369546472 shutdown_connections 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 --2- 172.21.15.28:0/1369546472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102ef0 0x7f91a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 --2- 172.21.15.28:0/1369546472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0102570 0x7f91a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 --2- 172.21.15.28:0/1369546472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0100e20 0x7f91a0101200 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 -- 172.21.15.28:0/1369546472 >> 172.21.15.28:0/1369546472 conn(0x7f91a00fc7d0 msgr2=0x7f91a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 -- 172.21.15.28:0/1369546472 shutdown_connections 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.860+0000 7f91a7c00640 1 -- 172.21.15.28:0/1369546472 wait complete. 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 Processor -- start 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 -- start start 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0100e20 0x7f91a0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102570 0x7f91a01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0110fa0 con 0x7f91a0102ef0 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0110e20 con 0x7f91a0102570 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a7c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0111120 con 0x7f91a0100e20 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a619d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a619d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53312/0 (socket says 172.21.15.28:53312) 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a619d640 1 -- 172.21.15.28:0/3705312116 learned_addr learned my addr 172.21.15.28:0/3705312116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:13.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a599c640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0100e20 0x7f91a0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.861+0000 7f91a519b640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102570 0x7f91a01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 -- 172.21.15.28:0/3705312116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0100e20 msgr2=0x7f91a0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0100e20 0x7f91a0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 -- 172.21.15.28:0/3705312116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102570 msgr2=0x7f91a01997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102570 0x7f91a01997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 -- 172.21.15.28:0/3705312116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a01a4830 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a619d640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f919800cce0 tx=0x7f919800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f9196ffd640 1 -- 172.21.15.28:0/3705312116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919800f450 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f9196ffd640 1 -- 172.21.15.28:0/3705312116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919800fad0 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a01a4a00 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f9196ffd640 1 -- 172.21.15.28:0/3705312116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198018b60 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.862+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a01a4ec0 con 0x7f91a0102ef0 2024-09-19T09:33:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.865+0000 7f9196ffd640 1 -- 172.21.15.28:0/3705312116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f919800fc80 con 0x7f91a0102ef0 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.865+0000 7f91a9fd9740 1 Processor -- start 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 -- start start 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 0x55716aef14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716aefeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55716ae532f0 con 0x55716aef19e0 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55716adfacf0 con 0x55716aefc780 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55716ae4f2b0 con 0x55716aef2f50 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53318/0 (socket says 172.21.15.28:53318) 2024-09-19T09:33:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 -- 172.21.15.28:0/1729467213 learned_addr learned my addr 172.21.15.28:0/1729467213 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a519b640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 0x55716aef14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a519b640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716aefeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f9196ffd640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 0x7f918c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a619d640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 0x7f918c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 -- 172.21.15.28:0/1729467213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 msgr2=0x55716aef14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 0x55716aef14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 -- 172.21.15.28:0/1729467213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 msgr2=0x55716aefeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716aefeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a599c640 1 -- 172.21.15.28:0/1729467213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55716addf3a0 con 0x55716aef19e0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.866+0000 7f91a519b640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 0x55716aef14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f9196ffd640 1 -- 172.21.15.28:0/3705312116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f91980cfe30 con 0x7f91a0102ef0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f91a599c640 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f919c009230 tx=0x7f919c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e71972d4d443f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f91a619d640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 0x7f918c0a5810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9198098cf0 tx=0x7f91980c70f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f918bfff640 1 -- 172.21.15.28:0/1729467213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c040070 con 0x55716aef19e0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f918bfff640 1 -- 172.21.15.28:0/1729467213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c0047a0 con 0x55716aef19e0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.867+0000 7f918bfff640 1 -- 172.21.15.28:0/1729467213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c035040 con 0x55716aef19e0 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- 172.21.15.28:0/1729467213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 msgr2=0x55716aef1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f919c009230 tx=0x7f919c004310 comp rx=0 tx=0).stop 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- 172.21.15.28:0/1729467213 shutdown_connections 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716aefeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef19e0 0x55716aef1e20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/1729467213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef2f50 0x55716aef14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- 172.21.15.28:0/1729467213 >> 172.21.15.28:0/1729467213 conn(0x55716aeb6310 msgr2=0x55716aeee070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- 172.21.15.28:0/1729467213 shutdown_connections 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- 172.21.15.28:0/1729467213 wait complete. 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 -- start start 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.868+0000 7f91a9fd9740 1 Processor -- start 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716aefc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55716aeff420 con 0x55716aef2f50 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55716ae0e060 con 0x55716aef19e0 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55716aeff5a0 con 0x55716aefc780 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a9fd9740 10 client.0 _ll_register_callbacks cb 0x55716ab618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53328/0 (socket says 172.21.15.28:53328) 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 -- 172.21.15.28:0/415324442 learned_addr learned my addr 172.21.15.28:0/415324442 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a519b640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a519b640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716aefc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 -- 172.21.15.28:0/415324442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 msgr2=0x55716aef9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 -- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 msgr2=0x55716aefc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716aefc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.869+0000 7f91a599c640 1 -- 172.21.15.28:0/415324442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55716abc43c0 con 0x55716aef2f50 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91a599c640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f919c0057c0 tx=0x7f919c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea8cff45f6fdb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91897fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c040070 con 0x55716aef2f50 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919c0047a0 con 0x55716aef2f50 2024-09-19T09:33:13.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f919c005230 con 0x55716aef2f50 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f919c055070 con 0x55716aef2f50 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91a9fd9740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55716af0aef0 con 0x55716aef2f50 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.870+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c0047a0 con 0x55716aef2f50 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f919c03c440 con 0x55716aef2f50 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 10 client.15540 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 20 client.15540 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af0c440 0x55716af0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 4 client.15540 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55716ae60860 con 0x55716af0c440 2024-09-19T09:33:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a519b640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af0c440 0x55716af0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91897fa640 10 client.15540 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:33:13.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.871+0000 7f91a519b640 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af0c440 0x55716af0e880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=492a74b05ffd5e2b server_cookie=ee99d460b0288c46 in_seq=0 out_seq=0 2024-09-19T09:33:13.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.872+0000 7f91897fa640 1 -- 172.21.15.28:0/415324442 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55716ae60860 con 0x55716af0c440 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.872+0000 7f91897fa640 10 client.15540 handle_command_reply: tid=0 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.872+0000 7f91a9fd9740 1 client.15540 shutdown 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.872+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 msgr2=0x55716aef9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.872+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f919c0057c0 tx=0x7f919c02a480 comp rx=0 tx=0).stop 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 shutdown_connections 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af0c440 0x55716af0e880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef9420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/415324442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716aefc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 >> 172.21.15.28:0/415324442 conn(0x55716aeb6310 msgr2=0x55716aef3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91897fa640 0 client.15540 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 shutdown_connections 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 1 -- 172.21.15.28:0/415324442 wait complete. 2024-09-19T09:33:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.873+0000 7f91a9fd9740 20 client.15540 trim_cache size 0 max 16384 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 Processor -- start 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 -- start start 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01ab140 con 0x55716aef2f50 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01acc80 con 0x55716aef19e0 2024-09-19T09:33:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01ab810 con 0x55716aefc780 2024-09-19T09:33:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a519b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a519b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:32822/0 (socket says 172.21.15.28:32822) 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a519b640 1 -- 172.21.15.28:0/647835150 learned_addr learned my addr 172.21.15.28:0/647835150 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a599c640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.887+0000 7f91a519b640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 -- 172.21.15.28:0/647835150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 msgr2=0x55716aef4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 -- 172.21.15.28:0/647835150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 msgr2=0x55716aef4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 -- 172.21.15.28:0/647835150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55716aef9d80 con 0x55716aef19e0 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a599c640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f91a519b640 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f919000ada0 tx=0x7f9190004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46571268e3d3574 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f918b7fe640 1 -- 172.21.15.28:0/647835150 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190032070 con 0x55716aef19e0 2024-09-19T09:33:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f918b7fe640 1 -- 172.21.15.28:0/647835150 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919000fdc0 con 0x55716aef19e0 2024-09-19T09:33:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.888+0000 7f918b7fe640 1 -- 172.21.15.28:0/647835150 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190020410 con 0x55716aef19e0 2024-09-19T09:33:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 -- 172.21.15.28:0/647835150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 msgr2=0x55716af086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f919000ada0 tx=0x7f9190004730 comp rx=0 tx=0).stop 2024-09-19T09:33:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 -- 172.21.15.28:0/647835150 shutdown_connections 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aefc780 0x55716aef4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aef4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/647835150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aef19e0 0x55716af086f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.889+0000 7f91a9fd9740 1 -- 172.21.15.28:0/647835150 >> 172.21.15.28:0/647835150 conn(0x55716aeb6310 msgr2=0x55716aef8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- 172.21.15.28:0/647835150 shutdown_connections 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- 172.21.15.28:0/647835150 wait complete. 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- start start 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 Processor -- start 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef19e0 0x55716aeff880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716af75150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01ac5b0 con 0x55716aef2f50 2024-09-19T09:33:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01abee0 con 0x55716aefc780 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.890+0000 7f91a9fd9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01a9140 con 0x55716aef19e0 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a9fd9740 10 client.0 _ll_register_callbacks cb 0x7f919000add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53344/0 (socket says 172.21.15.28:53344) 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 -- 172.21.15.28:0/3101902767 learned_addr learned my addr 172.21.15.28:0/3101902767 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a519b640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716af75150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a519b640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef19e0 0x55716aeff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 -- 172.21.15.28:0/3101902767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef19e0 msgr2=0x55716aeff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef19e0 0x55716aeff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 -- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 msgr2=0x55716af75150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716af75150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 -- 172.21.15.28:0/3101902767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55716af00670 con 0x55716aef2f50 2024-09-19T09:33:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91a599c640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f919c006010 tx=0x7f919c03c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fead09262b4c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.891+0000 7f91897fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c05e020 con 0x55716aef2f50 2024-09-19T09:33:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f919c004d60 con 0x55716aef2f50 2024-09-19T09:33:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919000fdc0 con 0x55716aef2f50 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f919c040040 con 0x55716aef2f50 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91a9fd9740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.892+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55716af766e0 con 0x55716aef2f50 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919000fdc0 con 0x55716aef2f50 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f919c031540 con 0x55716aef2f50 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 10 client.15543 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 20 client.15543 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af76e10 0x55716af79230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 4 client.15543 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:33:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55716ae5ad70 con 0x55716af76e10 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91a519b640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af76e10 0x55716af79230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.893+0000 7f91897fa640 10 client.15543 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a519b640 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af76e10 0x55716af79230 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=daef4c36bd2f9df6 server_cookie=244d53e3904852cc in_seq=0 out_seq=0 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91897fa640 1 -- 172.21.15.28:0/3101902767 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55716ae5ad70 con 0x55716af76e10 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91897fa640 10 client.15543 handle_command_reply: tid=0 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 client.15543 shutdown 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 msgr2=0x55716aeffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f919c006010 tx=0x7f919c03c6f0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 shutdown_connections 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55716af76e10 0x55716af79230 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55716aefc780 0x55716af75150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91897fa640 0 client.15543 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:33:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55716aef2f50 0x55716aeffdc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 --2- 172.21.15.28:0/3101902767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55716aef19e0 0x55716aeff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.894+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 >> 172.21.15.28:0/3101902767 conn(0x55716aeb6310 msgr2=0x55716aefac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 shutdown_connections 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a9fd9740 1 -- 172.21.15.28:0/3101902767 wait complete. 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a9fd9740 20 client.15543 trim_cache size 0 max 16384 2024-09-19T09:33:13.899 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 msgr2=0x7f918c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 0x7f918c0a5810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9198098cf0 tx=0x7f91980c70f0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 msgr2=0x7f91a01a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f919800cce0 tx=0x7f919800a620 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 shutdown_connections 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f918c0a3350 0x7f918c0a5810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91a0102570 0x7f91a01997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.895+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91a0102ef0 0x7f91a01a42f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.896+0000 7f91a7c00640 1 --2- 172.21.15.28:0/3705312116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a0100e20 0x7f91a0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.896+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 >> 172.21.15.28:0/3705312116 conn(0x7f91a00fc7d0 msgr2=0x7f91a01067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.896+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 shutdown_connections 2024-09-19T09:33:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:13.896+0000 7f91a7c00640 1 -- 172.21.15.28:0/3705312116 wait complete. 2024-09-19T09:33:14.677 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:14 smithi028 ceph-mon[30972]: pgmap v355: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-19T09:33:14.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:33:14.710 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:14 smithi078 ceph-mon[35619]: pgmap v355: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-19T09:33:14.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:14 smithi038 ceph-mon[35271]: pgmap v355: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-19T09:33:15.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 -- 172.21.15.28:0/3147778291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00101de0 msgr2=0x7f5a001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/3147778291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00101de0 0x7f5a001021c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009900 tx=0x7f59f002a410 comp rx=0 tx=0).stop 2024-09-19T09:33:15.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 -- 172.21.15.28:0/3147778291 shutdown_connections 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/3147778291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00106f10 0x7f5a0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/3147778291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00102790 0x7f5a001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/3147778291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00101de0 0x7f5a001021c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 -- 172.21.15.28:0/3147778291 >> 172.21.15.28:0/3147778291 conn(0x7f5a000fd880 msgr2=0x7f5a000ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 -- 172.21.15.28:0/3147778291 shutdown_connections 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 -- 172.21.15.28:0/3147778291 wait complete. 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.198+0000 7f5a05a5b640 1 Processor -- start 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 -- start start 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00101de0 0x7f5a000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00106f10 0x7f5a00074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00111f70 con 0x7f5a00102790 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00111df0 con 0x7f5a00101de0 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f5a05a5b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a001120f0 con 0x7f5a00106f10 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f59fe7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f59fe7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53370/0 (socket says 172.21.15.28:53370) 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f59fe7fc640 1 -- 172.21.15.28:0/2654191279 learned_addr learned my addr 172.21.15.28:0/2654191279 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f59ff7fe640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00106f10 0x7f5a00074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.199+0000 7f59feffd640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00101de0 0x7f5a000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 -- 172.21.15.28:0/2654191279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00106f10 msgr2=0x7f5a00074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00106f10 0x7f5a00074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 -- 172.21.15.28:0/2654191279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00101de0 msgr2=0x7f5a000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00101de0 0x7f5a000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 -- 172.21.15.28:0/2654191279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a001a6910 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f59fe7fc640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f59f400eab0 tx=0x7f59f400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f400ce40 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a001a6c00 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f4011950 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4010ae0 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.200+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a001a7190 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.201+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a001021c0 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.203+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f59f4011b00 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.203+0000 7f5a04a59640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 0x7f59e80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.204+0000 7f59feffd640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 0x7f59e80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.204+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f59f40cffc0 con 0x7f5a00102790 2024-09-19T09:33:15.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.204+0000 7f59feffd640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 0x7f59e80a5810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009230 tx=0x7f59f00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.210+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59f4016030 con 0x7f5a00102790 2024-09-19T09:33:15.246 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2714499146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 msgr2=0x7ffb3c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2714499146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c108e80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009ca0 tx=0x7ffb3402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2714499146 shutdown_connections 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2714499146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c101c30 0x7ffb3c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2714499146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c108e80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2714499146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c100a40 0x7ffb3c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2714499146 >> 172.21.15.28:0/2714499146 conn(0x7ffb3c0fc1c0 msgr2=0x7ffb3c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2714499146 shutdown_connections 2024-09-19T09:33:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.467+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2714499146 wait complete. 2024-09-19T09:33:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.468+0000 7ffb41ef1640 1 Processor -- start 2024-09-19T09:33:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.468+0000 7ffb41ef1640 1 -- start start 2024-09-19T09:33:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.468+0000 7ffb41ef1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 0x7ffb3c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.468+0000 7ffb41ef1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 0x7ffb3c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.468+0000 7ffb41ef1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb41ef1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c10c6f0 con 0x7ffb3c1025f0 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb41ef1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c10c570 con 0x7ffb3c100a40 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb41ef1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c10c870 con 0x7ffb3c101c30 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53398/0 (socket says 172.21.15.28:53398) 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2695992468 learned_addr learned my addr 172.21.15.28:0/2695992468 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 0x7ffb3c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3affd640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 0x7ffb3c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2695992468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 msgr2=0x7ffb3c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 0x7ffb3c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2695992468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 msgr2=0x7ffb3c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 0x7ffb3c06a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2695992468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c19fa90 con 0x7ffb3c1025f0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 0x7ffb3c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3affd640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 0x7ffb3c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb340319f0 tx=0x7ffb3402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.469+0000 7ffb38ff9640 1 -- 172.21.15.28:0/2695992468 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34003c70 con 0x7ffb3c1025f0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.470+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3c19fcc0 con 0x7ffb3c1025f0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.470+0000 7ffb38ff9640 1 -- 172.21.15.28:0/2695992468 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb34003e10 con 0x7ffb3c1025f0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.470+0000 7ffb38ff9640 1 -- 172.21.15.28:0/2695992468 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3402fd30 con 0x7ffb3c1025f0 2024-09-19T09:33:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.470+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3c1a01b0 con 0x7ffb3c1025f0 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.472+0000 7ffb44c91740 1 Processor -- start 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb38ff9640 1 -- 172.21.15.28:0/2695992468 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7ffb34004040 con 0x7ffb3c1025f0 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 -- start start 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fa4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fa9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3f082f0 con 0x55d5d3faa370 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3eb0270 con 0x55d5d3fa52e0 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb44c91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3f042b0 con 0x55d5d3fa49a0 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb3affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.473+0000 7ffb3affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53406/0 (socket says 172.21.15.28:53406) 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 -- 172.21.15.28:0/1040933702 learned_addr learned my addr 172.21.15.28:0/1040933702 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fa4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fa9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 -- 172.21.15.28:0/1040933702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 msgr2=0x55d5d3fa4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fa4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 -- 172.21.15.28:0/1040933702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 msgr2=0x55d5d3fa9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fa9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 -- 172.21.15.28:0/1040933702 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d3eaf340 con 0x55d5d3faa370 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fa9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb38ff9640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 0x7ffb240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 0x7ffb240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb30013610 tx=0x7ffb3000c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2720a66284952b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb38ff9640 1 -- 172.21.15.28:0/2695992468 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ffb340f0fd0 con 0x7ffb3c1025f0 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb21ffb640 1 -- 172.21.15.28:0/1040933702 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30019070 con 0x55d5d3faa370 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb21ffb640 1 -- 172.21.15.28:0/1040933702 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb30004230 con 0x55d5d3faa370 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.474+0000 7ffb21ffb640 1 -- 172.21.15.28:0/1040933702 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30011910 con 0x55d5d3faa370 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 -- 172.21.15.28:0/1040933702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 msgr2=0x55d5d3fac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb30013610 tx=0x7ffb3000c750 comp rx=0 tx=0).stop 2024-09-19T09:33:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 0x7ffb240a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005fd0 tx=0x7ffb2c00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 -- 172.21.15.28:0/1040933702 shutdown_connections 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fa9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fac830 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1040933702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fa4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 -- 172.21.15.28:0/1040933702 >> 172.21.15.28:0/1040933702 conn(0x55d5d3fa0010 msgr2=0x55d5d3fa0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 -- 172.21.15.28:0/1040933702 shutdown_connections 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.475+0000 7ffb44c91740 1 -- 172.21.15.28:0/1040933702 wait complete. 2024-09-19T09:33:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 -- start start 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 Processor -- start 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3faf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3fad340 con 0x55d5d3faa370 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3e91290 con 0x55d5d3fa52e0 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5d3fad4c0 con 0x55d5d3fa49a0 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb44c91740 10 client.0 _ll_register_callbacks cb 0x55d5d3ec2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53410/0 (socket says 172.21.15.28:53410) 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:32922/0 (socket says 172.21.15.28:32922) 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3bfff640 1 -- 172.21.15.28:0/1085235361 learned_addr learned my addr 172.21.15.28:0/1085235361 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.476+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3faf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 -- 172.21.15.28:0/1085235361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 msgr2=0x55d5d3faf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3faf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 -- 172.21.15.28:0/1085235361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 msgr2=0x55d5d3fb61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 -- 172.21.15.28:0/1085235361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d3c1b210 con 0x55d5d3fa49a0 2024-09-19T09:33:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3faf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb1affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55d5d3fa5d10 tx=0x7ffb2c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c203fef120a05c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c023070 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb30004230 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ffb2c004220 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb2c02e050 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.477+0000 7ffb44c91740 10 client.25171 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5d3ce7620 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb30004230 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffb2c00f040 con 0x55d5d3fa49a0 2024-09-19T09:33:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 10 client.25171 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 10 client.25171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 20 client.25171 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d3fba880 0x55d5d3fbcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 4 client.25171 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5d3f15a20 con 0x55d5d3fba880 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.478+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d3fba880 0x55d5d3fbcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.479+0000 7ffb1affd640 10 client.25171 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.479+0000 7ffb3affd640 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d3fba880 0x55d5d3fbcd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792906a3044318a4 server_cookie=46328fdbaa3c368d in_seq=0 out_seq=0 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.479+0000 7ffb1affd640 1 -- 172.21.15.28:0/1085235361 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5d3f15a20 con 0x55d5d3fba880 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.479+0000 7ffb1affd640 10 client.25171 handle_command_reply: tid=0 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 client.25171 shutdown 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 msgr2=0x55d5d3faec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55d5d3fa5d10 tx=0x7ffb2c004730 comp rx=0 tx=0).stop 2024-09-19T09:33:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 shutdown_connections 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3faf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d3fba880 0x55d5d3fbcd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb1affd640 0 client.25171 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 --2- 172.21.15.28:0/1085235361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faec20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 >> 172.21.15.28:0/1085235361 conn(0x55d5d3fa0010 msgr2=0x55d5d3fa1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 shutdown_connections 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 1 -- 172.21.15.28:0/1085235361 wait complete. 2024-09-19T09:33:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.480+0000 7ffb44c91740 20 client.25171 trim_cache size 0 max 16384 2024-09-19T09:33:15.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.497+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a0010a0f0 con 0x7f5a00102790 2024-09-19T09:33:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.497+0000 7f5a04a59640 1 -- 172.21.15.28:0/2654191279 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f59f409cc50 con 0x7f5a00102790 2024-09-19T09:33:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:33:15.499 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:33:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 Processor -- start 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 -- start start 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1a8f60 con 0x55d5d3faa370 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1aaaa0 con 0x55d5d3fa52e0 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb44c91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1a9630 con 0x55d5d3fa49a0 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53424/0 (socket says 172.21.15.28:53424) 2024-09-19T09:33:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2827400481 learned_addr learned my addr 172.21.15.28:0/2827400481 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3affd640 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.499+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2827400481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 msgr2=0x55d5d3faf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2827400481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 msgr2=0x55d5d3fb6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb3bfff640 1 -- 172.21.15.28:0/2827400481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d3fbd380 con 0x55d5d3faa370 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006010 tx=0x7ffb34009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b133b760afda42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb217fa640 1 -- 172.21.15.28:0/2827400481 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340d5070 con 0x55d5d3faa370 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb217fa640 1 -- 172.21.15.28:0/2827400481 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb34036040 con 0x55d5d3faa370 2024-09-19T09:33:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.500+0000 7ffb217fa640 1 -- 172.21.15.28:0/2827400481 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340c6240 con 0x55d5d3faa370 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 msgr2=0x7f59e80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 0x7f59e80a5810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009230 tx=0x7f59f00058d0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 msgr2=0x7f5a00074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f59f400eab0 tx=0x7f59f400ef70 comp rx=0 tx=0).stop 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 shutdown_connections 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f59e80a3350 0x7f59e80a5810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a00106f10 0x7f5a00074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a00102790 0x7f5a00074a30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 --2- 172.21.15.28:0/2654191279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a00101de0 0x7f5a000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 >> 172.21.15.28:0/2654191279 conn(0x7f5a000fd880 msgr2=0x7f5a000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 shutdown_connections 2024-09-19T09:33:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.500+0000 7f5a05a5b640 1 -- 172.21.15.28:0/2654191279 wait complete. 2024-09-19T09:33:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- 172.21.15.28:0/2827400481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 msgr2=0x55d5d3f9f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006010 tx=0x7ffb34009900 comp rx=0 tx=0).stop 2024-09-19T09:33:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- 172.21.15.28:0/2827400481 shutdown_connections 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3f9f0b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 --2- 172.21.15.28:0/2827400481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa49a0 0x55d5d3faf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- 172.21.15.28:0/2827400481 >> 172.21.15.28:0/2827400481 conn(0x55d5d3fa0010 msgr2=0x55d5d3f9e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- 172.21.15.28:0/2827400481 shutdown_connections 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- 172.21.15.28:0/2827400481 wait complete. 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.501+0000 7ffb44c91740 1 -- start start 2024-09-19T09:33:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 Processor -- start 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1aa3d0 con 0x55d5d3faa370 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1a9d00 con 0x55d5d3fa49a0 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb44c91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c1a6f40 con 0x55d5d3fa52e0 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53432/0 (socket says 172.21.15.28:53432) 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3bfff640 1 -- 172.21.15.28:0/185364548 learned_addr learned my addr 172.21.15.28:0/185364548 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:32898/0 (socket says 172.21.15.28:32898) 2024-09-19T09:33:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb44c91740 10 client.0 _ll_register_callbacks cb 0x7ffb2c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.502+0000 7ffb3affd640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 -- 172.21.15.28:0/185364548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 msgr2=0x55d5d3fb3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 -- 172.21.15.28:0/185364548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 msgr2=0x55d5d3fb5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 -- 172.21.15.28:0/185364548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d3fb5730 con 0x55d5d3fa52e0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3bfff640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb3affd640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb4110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55d5d3fbd5b0 tx=0x7ffb30004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741e96c037fdd480 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb1affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.503+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30019040 con 0x55d5d3fa52e0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb34036040 con 0x55d5d3fa52e0 2024-09-19T09:33:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ffb300037f0 con 0x55d5d3fa52e0 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb300039f0 con 0x55d5d3fa52e0 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 10 client.25177 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5d4029d50 con 0x55d5d3fa52e0 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb34036040 con 0x55d5d3fa52e0 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffb30034030 con 0x55d5d3fa52e0 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 10 client.25177 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 10 client.25177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 20 client.25177 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d402a390 0x55d5d402c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 4 client.25177 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.504+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5d3faa0f0 con 0x55d5d402a390 2024-09-19T09:33:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.505+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d402a390 0x55d5d402c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.505+0000 7ffb1affd640 10 client.25177 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.505+0000 7ffb3b7fe640 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d402a390 0x55d5d402c7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae282ee3e79ef9bd server_cookie=d5e5ebb5fc17b081 in_seq=0 out_seq=0 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb1affd640 1 -- 172.21.15.28:0/185364548 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5d3faa0f0 con 0x55d5d402a390 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb1affd640 10 client.25177 handle_command_reply: tid=0 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 client.25177 shutdown 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 msgr2=0x55d5d3fb4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb4110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55d5d3fbd5b0 tx=0x7ffb30004500 comp rx=0 tx=0).stop 2024-09-19T09:33:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 shutdown_connections 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d5d402a390 0x55d5d402c7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5d3faa370 0x55d5d3fb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5d3fa52e0 0x55d5d3fb4110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb1affd640 0 client.25177 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 --2- 172.21.15.28:0/185364548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5d3fa49a0 0x55d5d3fb3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 >> 172.21.15.28:0/185364548 conn(0x55d5d3fa0010 msgr2=0x55d5d3f9f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 shutdown_connections 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 1 -- 172.21.15.28:0/185364548 wait complete. 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.512+0000 7ffb44c91740 20 client.25177 trim_cache size 0 max 16384 2024-09-19T09:33:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:33:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:33:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:33:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:33:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 msgr2=0x7ffb240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 0x7ffb240a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005fd0 tx=0x7ffb2c00b2e0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 msgr2=0x7ffb3c19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb340319f0 tx=0x7ffb3402f450 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 shutdown_connections 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ffb240a32d0 0x7ffb240a5790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c1025f0 0x7ffb3c19f180 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3c101c30 0x7ffb3c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 --2- 172.21.15.28:0/2695992468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb3c100a40 0x7ffb3c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 >> 172.21.15.28:0/2695992468 conn(0x7ffb3c0fc1c0 msgr2=0x7ffb3c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 shutdown_connections 2024-09-19T09:33:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:15.513+0000 7ffb41ef1640 1 -- 172.21.15.28:0/2695992468 wait complete. 2024-09-19T09:33:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:15 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2654191279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:15 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2654191279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:15.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:15 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2654191279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:15.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 -- 172.21.15.28:0/2063746188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c102ef0 msgr2=0x7f963c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 --2- 172.21.15.28:0/2063746188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c102ef0 0x7f963c10da40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9634009ca0 tx=0x7f963402a950 comp rx=0 tx=0).stop 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 -- 172.21.15.28:0/2063746188 shutdown_connections 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 --2- 172.21.15.28:0/2063746188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 0x7f963c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 --2- 172.21.15.28:0/2063746188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c102ef0 0x7f963c10da40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 --2- 172.21.15.28:0/2063746188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c100e20 0x7f963c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 -- 172.21.15.28:0/2063746188 >> 172.21.15.28:0/2063746188 conn(0x7f963c0fc7d0 msgr2=0x7f963c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 -- 172.21.15.28:0/2063746188 shutdown_connections 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.958+0000 7f9640f03640 1 -- 172.21.15.28:0/2063746188 wait complete. 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 Processor -- start 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 -- start start 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 0x7f963c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 0x7f963c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110f80 con 0x7f963c100e20 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110e00 con 0x7f963c102570 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9640f03640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963c111100 con 0x7f963c102ef0 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f963a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f963a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53448/0 (socket says 172.21.15.28:53448) 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f963a59c640 1 -- 172.21.15.28:0/4248649911 learned_addr learned my addr 172.21.15.28:0/4248649911 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.959+0000 7f9639d9b640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 0x7f963c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963ad9d640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 0x7f963c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 -- 172.21.15.28:0/4248649911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 msgr2=0x7f963c199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 0x7f963c199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 -- 172.21.15.28:0/4248649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 msgr2=0x7f963c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 0x7f963c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 -- 172.21.15.28:0/4248649911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a47d0 con 0x7f963c100e20 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963ad9d640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 0x7f963c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f963a59c640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e9f0 tx=0x7f962c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f962b7fe640 1 -- 172.21.15.28:0/4248649911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00cd80 con 0x7f963c100e20 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f962b7fe640 1 -- 172.21.15.28:0/4248649911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c005500 con 0x7f963c100e20 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a4ac0 con 0x7f963c100e20 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f962b7fe640 1 -- 172.21.15.28:0/4248649911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c010690 con 0x7f963c100e20 2024-09-19T09:33:15.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.960+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a5000 con 0x7f963c100e20 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.963+0000 7f962b7fe640 1 -- 172.21.15.28:0/4248649911 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f962c0046c0 con 0x7f963c100e20 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.963+0000 7f96432dc740 1 Processor -- start 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.963+0000 7f96432dc740 1 -- start start 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f962b7fe640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 0x7f96240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c205d0 0x55a0d5c20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5b862f0 con 0x55a0d5c2b370 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5b2dcf0 con 0x55a0d5c21b40 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f96432dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5b822b0 con 0x55a0d5c205d0 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f962b7fe640 1 -- 172.21.15.28:0/4248649911 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f962c0cfea0 con 0x7f963c100e20 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963ad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:32914/0 (socket says 172.21.15.28:32914) 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963ad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53452/0 (socket says 172.21.15.28:53452) 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 -- 172.21.15.28:0/1576690607 learned_addr learned my addr 172.21.15.28:0/1576690607 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f9639d9b640 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c205d0 0x55a0d5c20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963ad9d640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 0x7f96240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 -- 172.21.15.28:0/1576690607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c205d0 msgr2=0x55a0d5c20a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c205d0 0x55a0d5c20a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 -- 172.21.15.28:0/1576690607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 msgr2=0x55a0d5c2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.964+0000 7f963a59c640 1 -- 172.21.15.28:0/1576690607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d5b123a0 con 0x55a0d5c21b40 2024-09-19T09:33:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.965+0000 7f963a59c640 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f962c009380 tx=0x7f962c09c290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19393f85964b944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.965+0000 7f963ad9d640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 0x7f96240a5810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9634005fc0 tx=0x7f963402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.965+0000 7f9628ff9640 1 -- 172.21.15.28:0/1576690607 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0af0b0 con 0x55a0d5c21b40 2024-09-19T09:33:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.965+0000 7f9628ff9640 1 -- 172.21.15.28:0/1576690607 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c09ca90 con 0x55a0d5c21b40 2024-09-19T09:33:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.965+0000 7f9628ff9640 1 -- 172.21.15.28:0/1576690607 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0aeed0 con 0x55a0d5c21b40 2024-09-19T09:33:15.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- 172.21.15.28:0/1576690607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 msgr2=0x55a0d5c20090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f962c009380 tx=0x7f962c09c290 comp rx=0 tx=0).stop 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- 172.21.15.28:0/1576690607 shutdown_connections 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c205d0 0x55a0d5c20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 --2- 172.21.15.28:0/1576690607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c21b40 0x55a0d5c20090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- 172.21.15.28:0/1576690607 >> 172.21.15.28:0/1576690607 conn(0x55a0d5be9310 msgr2=0x55a0d5c1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- 172.21.15.28:0/1576690607 shutdown_connections 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- 172.21.15.28:0/1576690607 wait complete. 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 -- start start 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.966+0000 7f96432dc740 1 Processor -- start 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c2ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5c2e070 con 0x55a0d5c21b40 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5b41060 con 0x55a0d5c205d0 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d5c2e1f0 con 0x55a0d5c2b370 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f96432dc740 10 client.0 _ll_register_callbacks cb 0x55a0d58948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53462/0 (socket says 172.21.15.28:53462) 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 -- 172.21.15.28:0/1607397480 learned_addr learned my addr 172.21.15.28:0/1607397480 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f963a59c640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c2ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f963ad9d640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 -- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 msgr2=0x55a0d5c28650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c28650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 -- 172.21.15.28:0/1607397480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 msgr2=0x55a0d5c2ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c2ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.967+0000 7f9639d9b640 1 -- 172.21.15.28:0/1607397480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d58f73c0 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9639d9b640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9630015b00 tx=0x7f96300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d823c278c6ff69d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963000be50 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9630005020 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962c09ca90 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9630026070 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f96432dc740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d5c39ba0 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c09ca90 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9630004810 con 0x55a0d5c21b40 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f96432dc740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.968+0000 7f96432dc740 10 client.15579 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:33:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f96432dc740 20 client.15579 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5c3b060 0x55a0d5c3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f96432dc740 4 client.15579 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d5b93860 con 0x55a0d5c3b060 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f963a59c640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5c3b060 0x55a0d5c3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f963a59c640 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5c3b060 0x55a0d5c3d4b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ea270801a87beba server_cookie=f68bea275d47b4f3 in_seq=0 out_seq=0 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.969+0000 7f9621ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f9621ffb640 1 -- 172.21.15.28:0/1607397480 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0d5b93860 con 0x55a0d5c3b060 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f9621ffb640 10 client.15579 handle_command_reply: tid=0 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 client.15579 shutdown 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 msgr2=0x55a0d5c28040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9630015b00 tx=0x7f96300045e0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 shutdown_connections 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5c3b060 0x55a0d5c3d4b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c28040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 --2- 172.21.15.28:0/1607397480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c2ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 >> 172.21.15.28:0/1607397480 conn(0x55a0d5be9310 msgr2=0x55a0d5c229c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f9621ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 shutdown_connections 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 1 -- 172.21.15.28:0/1607397480 wait complete. 2024-09-19T09:33:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.970+0000 7f96432dc740 20 client.15579 trim_cache size 0 max 16384 2024-09-19T09:33:15.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 Processor -- start 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 -- start start 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c1ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ab360 con 0x55a0d5c21b40 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1acea0 con 0x55a0d5c205d0 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f96432dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1aba30 con 0x55a0d5c2b370 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53478/0 (socket says 172.21.15.28:53478) 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 -- 172.21.15.28:0/938030834 learned_addr learned my addr 172.21.15.28:0/938030834 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f963a59c640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c1ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 -- 172.21.15.28:0/938030834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 msgr2=0x55a0d5c231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f963ad9d640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 -- 172.21.15.28:0/938030834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 msgr2=0x55a0d5c1ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c1ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f9639d9b640 1 -- 172.21.15.28:0/938030834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d5c26590 con 0x55a0d5c21b40 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.984+0000 7f963a59c640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c1ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.985+0000 7f9639d9b640 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9630004e20 tx=0x7f963000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f9b0b3ed9cf959 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.985+0000 7f9623fff640 1 -- 172.21.15.28:0/938030834 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630004650 con 0x55a0d5c21b40 2024-09-19T09:33:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.985+0000 7f9623fff640 1 -- 172.21.15.28:0/938030834 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96300047f0 con 0x55a0d5c21b40 2024-09-19T09:33:15.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.985+0000 7f9623fff640 1 -- 172.21.15.28:0/938030834 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300052c0 con 0x55a0d5c21b40 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- 172.21.15.28:0/938030834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 msgr2=0x55a0d5c37310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9630004e20 tx=0x7f963000bed0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- 172.21.15.28:0/938030834 shutdown_connections 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c2b370 0x55a0d5c231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c21b40 0x55a0d5c37310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- 172.21.15.28:0/938030834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c1ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- 172.21.15.28:0/938030834 >> 172.21.15.28:0/938030834 conn(0x55a0d5be9310 msgr2=0x55a0d5c1c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- 172.21.15.28:0/938030834 shutdown_connections 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- 172.21.15.28:0/938030834 wait complete. 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 -- start start 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 Processor -- start 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.986+0000 7f96432dc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c21b40 0x55a0d5c339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f96432dc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f96432dc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ac7d0 con 0x55a0d5c2b370 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f96432dc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ac100 con 0x55a0d5c205d0 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f96432dc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1a9340 con 0x55a0d5c21b40 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f96432dc740 10 client.0 _ll_register_callbacks cb 0x7f9630004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963ad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:32946/0 (socket says 172.21.15.28:32946) 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963ad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53494/0 (socket says 172.21.15.28:53494) 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 -- 172.21.15.28:0/2238182807 learned_addr learned my addr 172.21.15.28:0/2238182807 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f9639d9b640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c21b40 0x55a0d5c339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 -- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c21b40 msgr2=0x55a0d5c339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c21b40 0x55a0d5c339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 -- 172.21.15.28:0/2238182807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 msgr2=0x55a0d5c24650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 -- 172.21.15.28:0/2238182807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d5c24cc0 con 0x55a0d5c205d0 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963ad9d640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f963a59c640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cde00 tx=0x7f962c0056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e688214e4c684926 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.987+0000 7f9621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:33:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c086e60 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f962c098a00 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96300047f0 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f962c09ca90 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 10 client.25250 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d5ca5620 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c0a0740 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f962c0a9020 con 0x55a0d5c205d0 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 10 client.25250 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 20 client.25250 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5ca5d00 0x55a0d5ca80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 4 client.25250 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.988+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f96300047f0 con 0x55a0d5ca5d00 2024-09-19T09:33:15.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9639d9b640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5ca5d00 0x55a0d5ca80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9621ffb640 10 client.25250 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9639d9b640 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5ca5d00 0x55a0d5ca80f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e97a51c99539825f server_cookie=8e6ccf87d9a8f2f3 in_seq=0 out_seq=0 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9621ffb640 1 -- 172.21.15.28:0/2238182807 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f96300047f0 con 0x55a0d5ca5d00 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9621ffb640 10 client.25250 handle_command_reply: tid=0 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 client.25250 shutdown 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 msgr2=0x55a0d5c33480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cde00 tx=0x7f962c0056b0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 shutdown_connections 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55a0d5ca5d00 0x55a0d5ca80f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a0d5c2b370 0x55a0d5c24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d5c21b40 0x55a0d5c339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.989+0000 7f9621ffb640 0 client.25250 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f96432dc740 1 --2- 172.21.15.28:0/2238182807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0d5c205d0 0x55a0d5c33480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 >> 172.21.15.28:0/2238182807 conn(0x55a0d5be9310 msgr2=0x55a0d5c298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 shutdown_connections 2024-09-19T09:33:15.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f96432dc740 1 -- 172.21.15.28:0/2238182807 wait complete. 2024-09-19T09:33:15.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f96432dc740 20 client.25250 trim_cache size 0 max 16384 2024-09-19T09:33:15.996 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 msgr2=0x7f96240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 0x7f96240a5810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9634005fc0 tx=0x7f963402a950 comp rx=0 tx=0).stop 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 msgr2=0x7f963c1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e9f0 tx=0x7f962c00edf0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 shutdown_connections 2024-09-19T09:33:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f96240a3350 0x7f96240a5810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f963c102570 0x7f963c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f963c102ef0 0x7f963c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 --2- 172.21.15.28:0/4248649911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f963c100e20 0x7f963c1993d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 >> 172.21.15.28:0/4248649911 conn(0x7f963c0fc7d0 msgr2=0x7f963c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 shutdown_connections 2024-09-19T09:33:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:15.990+0000 7f9640f03640 1 -- 172.21.15.28:0/4248649911 wait complete. 2024-09-19T09:33:16.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:16 smithi028 ceph-mon[30972]: pgmap v356: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:33:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:16 smithi078 ceph-mon[35619]: pgmap v356: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:33:16.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:16 smithi038 ceph-mon[35271]: pgmap v356: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:33:18.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:18 smithi078 ceph-mon[35619]: pgmap v357: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:33:18.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:18 smithi038 ceph-mon[35271]: pgmap v357: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:33:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:18 smithi028 ceph-mon[30972]: pgmap v357: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:33:20.120 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T09:33:20.121 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T09:33:20.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:20 smithi078 ceph-mon[35619]: pgmap v358: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:33:20.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:20 smithi038 ceph-mon[35271]: pgmap v358: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:33:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:20 smithi028 ceph-mon[30972]: pgmap v358: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:33:21.097 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:33:21.153 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 -- 172.21.15.28:0/1828338229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 msgr2=0x7fb9bc1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 --2- 172.21.15.28:0/1828338229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1057e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009900 tx=0x7fb9ac02a3f0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 -- 172.21.15.28:0/1828338229 shutdown_connections 2024-09-19T09:33:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 --2- 172.21.15.28:0/1828338229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc100090 0x7fb9bc100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 --2- 172.21.15.28:0/1828338229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1057e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 --2- 172.21.15.28:0/1828338229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc106b70 0x7fb9bc106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 -- 172.21.15.28:0/1828338229 >> 172.21.15.28:0/1828338229 conn(0x7fb9bc077210 msgr2=0x7fb9bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 -- 172.21.15.28:0/1828338229 shutdown_connections 2024-09-19T09:33:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.655+0000 7fb9c087f640 1 -- 172.21.15.28:0/1828338229 wait complete. 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 Processor -- start 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 -- start start 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc100090 0x7fb9bc1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc106b70 0x7fb9bc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc118580 con 0x7fb9bc100a50 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc118400 con 0x7fb9bc106b70 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.656+0000 7fb9c087f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc118700 con 0x7fb9bc100090 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc106b70 0x7fb9bc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53504/0 (socket says 172.21.15.28:53504) 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2729369288 learned_addr learned my addr 172.21.15.28:0/2729369288 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc100090 0x7fb9bc1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2729369288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc100090 msgr2=0x7fb9bc1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc100090 0x7fb9bc1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2729369288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc106b70 msgr2=0x7fb9bc19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc106b70 0x7fb9bc19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2729369288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc19ef30 con 0x7fb9bc100a50 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.657+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.658+0000 7fb9b37fe640 1 -- 172.21.15.28:0/2729369288 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400cd80 con 0x7fb9bc100a50 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.658+0000 7fb9b37fe640 1 -- 172.21.15.28:0/2729369288 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4010910 con 0x7fb9bc100a50 2024-09-19T09:33:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.658+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc19f220 con 0x7fb9bc100a50 2024-09-19T09:33:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.658+0000 7fb9b37fe640 1 -- 172.21.15.28:0/2729369288 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4018990 con 0x7fb9bc100a50 2024-09-19T09:33:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.658+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1ab470 con 0x7fb9bc100a50 2024-09-19T09:33:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.661+0000 7fb9c2c58740 1 Processor -- start 2024-09-19T09:33:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 -- start start 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c4a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 0x556ff3c4cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ff3ba52a0 con 0x556ff3c4a940 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ff3b4cb10 con 0x556ff3c3f760 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ff3ba1260 con 0x556ff3c40080 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9b37fe640 1 -- 172.21.15.28:0/2729369288 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fb9a4018b50 con 0x7fb9bc100a50 2024-09-19T09:33:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c4a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9bad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:32964/0 (socket says 172.21.15.28:32964) 2024-09-19T09:33:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9bad9d640 1 -- 172.21.15.28:0/3454423545 learned_addr learned my addr 172.21.15.28:0/3454423545 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.662+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 0x556ff3c4cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3454423545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 msgr2=0x556ff3c4a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c4a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3454423545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 msgr2=0x556ff3c3fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3454423545 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff3916b90 con 0x556ff3c4a940 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 0x556ff3c4cd30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cd750 tx=0x7fb9a40d84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15b71092f499e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.663+0000 7fb9b37fe640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 0x7fb9a80a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.664+0000 7fb9b37fe640 1 -- 172.21.15.28:0/2729369288 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb9a40cfd70 con 0x7fb9bc100a50 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.664+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 0x7fb9a80a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.664+0000 7fb9b0ff9640 1 -- 172.21.15.28:0/3454423545 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40e3070 con 0x556ff3c4a940 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.664+0000 7fb9b0ff9640 1 -- 172.21.15.28:0/3454423545 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a409c960 con 0x556ff3c4a940 2024-09-19T09:33:21.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.664+0000 7fb9b0ff9640 1 -- 172.21.15.28:0/3454423545 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409c260 con 0x556ff3c4a940 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3454423545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 msgr2=0x556ff3c4cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 0x556ff3c4cd30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cd750 tx=0x7fb9a40d84b0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 0x7fb9a80a58e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac033000 tx=0x7fb9ac005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3454423545 shutdown_connections 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c4a940 0x556ff3c4cd30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c4a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3454423545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c3f760 0x556ff3c3fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.665+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3454423545 >> 172.21.15.28:0/3454423545 conn(0x556ff3c3bcf0 msgr2=0x556ff3c3c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.666+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3454423545 shutdown_connections 2024-09-19T09:33:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.666+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3454423545 wait complete. 2024-09-19T09:33:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.666+0000 7fb9c2c58740 1 -- start start 2024-09-19T09:33:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.666+0000 7fb9c2c58740 1 Processor -- start 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ff3c4d510 con 0x556ff3c40080 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ff3b2d6e0 con 0x556ff3c4a940 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ff3c4d690 con 0x556ff3c3f760 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9c2c58740 10 client.0 _ll_register_callbacks cb 0x556ff3b51a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53508/0 (socket says 172.21.15.28:53508) 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3067887867 learned_addr learned my addr 172.21.15.28:0/3067887867 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 msgr2=0x556ff3c478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3067887867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 msgr2=0x556ff3c48330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.667+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/3067887867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff3c56a80 con 0x556ff3c40080 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cdcd0 tx=0x7fb9a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f7fff2a9ad27c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4098500 con 0x556ff3c40080 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a409c960 con 0x556ff3c40080 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb9a40d39d0 con 0x556ff3c40080 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9a409bda0 con 0x556ff3c40080 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9c2c58740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.668+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ff3c4d510 con 0x556ff3c40080 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40a0610 con 0x556ff3c40080 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9a40988d0 con 0x556ff3c40080 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 10 client.15597 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 20 client.15597 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3c5a590 0x556ff3c5c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 4 client.15597 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.669+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9a409c960 con 0x556ff3c5a590 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.670+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3c5a590 0x556ff3c5c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.670+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3c5a590 0x556ff3c5c980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d68f018534ccaa server_cookie=4e92d5dc59e2fe6f in_seq=0 out_seq=0 2024-09-19T09:33:21.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.670+0000 7fb9a1ffb640 10 client.15597 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/3067887867 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb9a409c960 con 0x556ff3c5a590 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9a1ffb640 10 client.15597 handle_command_reply: tid=0 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 client.15597 shutdown 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 msgr2=0x556ff3c47df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cdcd0 tx=0x7fb9a4009310 comp rx=0 tx=0).stop 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 shutdown_connections 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3c5a590 0x556ff3c5c980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c47df0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/3067887867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 >> 172.21.15.28:0/3067887867 conn(0x556ff3c3bcf0 msgr2=0x556ff3c419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9a1ffb640 0 client.15597 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 shutdown_connections 2024-09-19T09:33:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 1 -- 172.21.15.28:0/3067887867 wait complete. 2024-09-19T09:33:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.671+0000 7fb9c2c58740 20 client.15597 trim_cache size 0 max 16384 2024-09-19T09:33:21.683 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:21.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.694+0000 7fb9c2c58740 1 Processor -- start 2024-09-19T09:33:21.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 -- start start 2024-09-19T09:33:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9ba59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9ba59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53524/0 (socket says 172.21.15.28:53524) 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b3b60 con 0x556ff3c3f760 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b56a0 con 0x556ff3c4a940 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b4230 con 0x556ff3c40080 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:33042/0 (socket says 172.21.15.28:33042) 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2390327744 learned_addr learned my addr 172.21.15.28:0/2390327744 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.695+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2390327744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 msgr2=0x556ff3c48180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2390327744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 msgr2=0x556ff3c46bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/2390327744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff3c37260 con 0x556ff3c40080 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40b52b0 tx=0x7fb9a409bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0c1e3bbafad44f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9a3fff640 1 -- 172.21.15.28:0/2390327744 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409c030 con 0x556ff3c40080 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9a3fff640 1 -- 172.21.15.28:0/2390327744 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a409c960 con 0x556ff3c40080 2024-09-19T09:33:21.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.696+0000 7fb9a3fff640 1 -- 172.21.15.28:0/2390327744 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40d39b0 con 0x556ff3c40080 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.697+0000 7fb9c2c58740 1 -- 172.21.15.28:0/2390327744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 msgr2=0x556ff3c49780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.697+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40b52b0 tx=0x7fb9a409bbc0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 -- 172.21.15.28:0/2390327744 shutdown_connections 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c48180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c40080 0x556ff3c49780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/2390327744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c3f760 0x556ff3c46bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 -- 172.21.15.28:0/2390327744 >> 172.21.15.28:0/2390327744 conn(0x556ff3c3bcf0 msgr2=0x556ff3c3a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 -- 172.21.15.28:0/2390327744 shutdown_connections 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.698+0000 7fb9c2c58740 1 -- 172.21.15.28:0/2390327744 wait complete. 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.699+0000 7fb9c2c58740 1 -- start start 2024-09-19T09:33:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.699+0000 7fb9c2c58740 1 Processor -- start 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.699+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c5fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9c2c58740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c5d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c5fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9b9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53536/0 (socket says 172.21.15.28:53536) 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c5fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:33058/0 (socket says 172.21.15.28:33058) 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9c2c58740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b4fd0 con 0x556ff3c40080 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/1290194610 learned_addr learned my addr 172.21.15.28:0/1290194610 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b4900 con 0x556ff3c4a940 2024-09-19T09:33:21.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c5d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.700+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b1b40 con 0x556ff3c3f760 2024-09-19T09:33:21.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9c2c58740 10 client.0 _ll_register_callbacks cb 0x7fb9a40d3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:21.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 msgr2=0x556ff3c5fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c5fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/1290194610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 msgr2=0x556ff3c5d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c5d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff3c5daf0 con 0x556ff3c40080 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9bad9d640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c5d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9b9d9b640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40d8d00 tx=0x7fb9a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ad748a92c7376e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.701+0000 7fb9a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40ba060 con 0x556ff3c40080 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fb9a40a4c40 con 0x556ff3c40080 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a409c960 con 0x556ff3c40080 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9a409c030 con 0x556ff3c40080 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9c2c58740 10 client.15600 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:21.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.702+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ff3c5daf0 con 0x556ff3c40080 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a409c960 con 0x556ff3c40080 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9a40aa020 con 0x556ff3c40080 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 10 client.15600 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 20 client.15600 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3cc4fb0 0x556ff3cc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 4 client.15600 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.703+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x556ff3bb29d0 con 0x556ff3cc4fb0 2024-09-19T09:33:21.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.704+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3cc4fb0 0x556ff3cc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:21.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.704+0000 7fb9a1ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:21.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.704+0000 7fb9ba59c640 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3cc4fb0 0x556ff3cc73a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04d35eb58ec8f24 server_cookie=10e5aef66aed09bd in_seq=0 out_seq=0 2024-09-19T09:33:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9a1ffb640 1 -- 172.21.15.28:0/1290194610 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556ff3bb29d0 con 0x556ff3cc4fb0 2024-09-19T09:33:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9a1ffb640 10 client.15600 handle_command_reply: tid=0 2024-09-19T09:33:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 client.15600 shutdown 2024-09-19T09:33:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 msgr2=0x556ff3c5cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40d8d00 tx=0x7fb9a4009310 comp rx=0 tx=0).stop 2024-09-19T09:33:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 shutdown_connections 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ff3c4a940 0x556ff3c5d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ff3cc4fb0 0x556ff3cc73a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ff3c40080 0x556ff3c5cf10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 --2- 172.21.15.28:0/1290194610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ff3c3f760 0x556ff3c5fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 >> 172.21.15.28:0/1290194610 conn(0x556ff3c3bcf0 msgr2=0x556ff3c411b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.718+0000 7fb9a1ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 shutdown_connections 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c2c58740 1 -- 172.21.15.28:0/1290194610 wait complete. 2024-09-19T09:33:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c2c58740 20 client.15600 trim_cache size 0 max 16384 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 2, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "b7b0c68d": { 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 2, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:33:21.722 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 131.2, 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:21.723 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 msgr2=0x7fb9a80a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 0x7fb9a80a58e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac033000 tx=0x7fb9ac005de0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 msgr2=0x7fb9bc1a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.719+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 shutdown_connections 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb9a80a3420 0x7fb9a80a58e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9bc106b70 0x7fb9bc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9bc100a50 0x7fb9bc1a4b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 --2- 172.21.15.28:0/2729369288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9bc100090 0x7fb9bc1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 >> 172.21.15.28:0/2729369288 conn(0x7fb9bc077210 msgr2=0x7fb9bc1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 shutdown_connections 2024-09-19T09:33:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:21.720+0000 7fb9c087f640 1 -- 172.21.15.28:0/2729369288 wait complete. 2024-09-19T09:33:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 -- 172.21.15.28:0/3695395461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841082a0 msgr2=0x7fc48410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 --2- 172.21.15.28:0/3695395461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841082a0 0x7fc48410ec40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009cd0 tx=0x7fc47c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 -- 172.21.15.28:0/3695395461 shutdown_connections 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 --2- 172.21.15.28:0/3695395461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841082a0 0x7fc48410ec40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 --2- 172.21.15.28:0/3695395461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4841036e0 0x7fc484107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 --2- 172.21.15.28:0/3695395461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc484102d30 0x7fc484103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 -- 172.21.15.28:0/3695395461 >> 172.21.15.28:0/3695395461 conn(0x7fc4840fc7d0 msgr2=0x7fc4840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 -- 172.21.15.28:0/3695395461 shutdown_connections 2024-09-19T09:33:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.498+0000 7fc48adbb640 1 -- 172.21.15.28:0/3695395461 wait complete. 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 Processor -- start 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 -- start start 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc484102d30 0x7fc484198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 0x7fc48419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841120d0 con 0x7fc4841036e0 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc484111f50 con 0x7fc4841082a0 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc48adbb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc484112250 con 0x7fc484102d30 2024-09-19T09:33:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc483fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc483fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53548/0 (socket says 172.21.15.28:53548) 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc483fff640 1 -- 172.21.15.28:0/1656505696 learned_addr learned my addr 172.21.15.28:0/1656505696 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.499+0000 7fc488b57640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc484102d30 0x7fc484198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc489358640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 0x7fc48419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 -- 172.21.15.28:0/1656505696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc484102d30 msgr2=0x7fc484198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc484102d30 0x7fc484198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 -- 172.21.15.28:0/1656505696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 msgr2=0x7fc48419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 0x7fc48419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 -- 172.21.15.28:0/1656505696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48419d5c0 con 0x7fc4841036e0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc489358640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 0x7fc48419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc483fff640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d740 tx=0x7fc47000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc481ffb640 1 -- 172.21.15.28:0/1656505696 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470017070 con 0x7fc4841036e0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc481ffb640 1 -- 172.21.15.28:0/1656505696 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc47000de50 con 0x7fc4841036e0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.500+0000 7fc481ffb640 1 -- 172.21.15.28:0/1656505696 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47000f6b0 con 0x7fc4841036e0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.501+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48419d8b0 con 0x7fc4841036e0 2024-09-19T09:33:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.501+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841a5190 con 0x7fc4841036e0 2024-09-19T09:33:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc481ffb640 1 -- 172.21.15.28:0/1656505696 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fc47001a070 con 0x7fc4841036e0 2024-09-19T09:33:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 Processor -- start 2024-09-19T09:33:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 -- start start 2024-09-19T09:33:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c716c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c715560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c72af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56536c67b2f0 con 0x56536c715b80 2024-09-19T09:33:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56536c622cf0 con 0x56536c715120 2024-09-19T09:33:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.505+0000 7fc48d194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56536c6772b0 con 0x56536c716840 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.506+0000 7fc489358640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c716c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.506+0000 7fc489358640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c716c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:33086/0 (socket says 172.21.15.28:33086) 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.506+0000 7fc489358640 1 -- 172.21.15.28:0/2524258524 learned_addr learned my addr 172.21.15.28:0/2524258524 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.506+0000 7fc488b57640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c715560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.506+0000 7fc483fff640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c72af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc481ffb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 0x7fc4740a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 -- 172.21.15.28:0/2524258524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 msgr2=0x56536c716c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c716c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 -- 172.21.15.28:0/2524258524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 msgr2=0x56536c72af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c72af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 -- 172.21.15.28:0/2524258524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56536c6073a0 con 0x56536c715120 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc483fff640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c72af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc481ffb640 1 -- 172.21.15.28:0/1656505696 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc4700cf140 con 0x7fc4841036e0 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc489358640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 0x7fc4740a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.507+0000 7fc488b57640 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c715560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc478009c50 tx=0x7fc478011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a3f71e76dabf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.508+0000 7fc46effd640 1 -- 172.21.15.28:0/2524258524 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004050 con 0x56536c715120 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.508+0000 7fc46effd640 1 -- 172.21.15.28:0/2524258524 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478005500 con 0x56536c715120 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.508+0000 7fc46effd640 1 -- 172.21.15.28:0/2524258524 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004e40 con 0x56536c715120 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- 172.21.15.28:0/2524258524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 msgr2=0x56536c715560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc489358640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 0x7fc4740a5810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009430 tx=0x7fc47c03f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c715560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc478009c50 tx=0x7fc478011810 comp rx=0 tx=0).stop 2024-09-19T09:33:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- 172.21.15.28:0/2524258524 shutdown_connections 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c715560 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c72af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 --2- 172.21.15.28:0/2524258524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c716c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- 172.21.15.28:0/2524258524 >> 172.21.15.28:0/2524258524 conn(0x56536c6de310 msgr2=0x56536c711c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- 172.21.15.28:0/2524258524 shutdown_connections 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- 172.21.15.28:0/2524258524 wait complete. 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.509+0000 7fc48d194740 1 -- start start 2024-09-19T09:33:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 Processor -- start 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c7280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c728600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56536c72b930 con 0x56536c715b80 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56536c636060 con 0x56536c715120 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56536c72bab0 con 0x56536c716840 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc488b57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc48d194740 10 client.0 _ll_register_callbacks cb 0x56536c3898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:33024/0 (socket says 172.21.15.28:33024) 2024-09-19T09:33:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 -- 172.21.15.28:0/40631402 learned_addr learned my addr 172.21.15.28:0/40631402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.510+0000 7fc483fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c7280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc489358640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c728600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 -- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 msgr2=0x56536c728600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c728600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 -- 172.21.15.28:0/40631402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 msgr2=0x56536c7280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c7280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 -- 172.21.15.28:0/40631402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56536c3ec3c0 con 0x56536c715120 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc488b57640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc47801b040 tx=0x7fc478004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ed5c7680f11dca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc463fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478027070 con 0x56536c715120 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.511+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc478005500 con 0x56536c715120 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc478004d80 con 0x56536c715120 2024-09-19T09:33:22.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc478033050 con 0x56536c715120 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 10 client.25277 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56536c7372f0 con 0x56536c715120 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478005500 con 0x56536c715120 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc478003c60 con 0x56536c715120 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 10 client.25277 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 10 client.25277 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 20 client.25277 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c738ae0 0x56536c73af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 4 client.25277 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.512+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56536c688860 con 0x56536c738ae0 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.513+0000 7fc483fff640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c738ae0 0x56536c73af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.513+0000 7fc463fff640 10 client.25277 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.514+0000 7fc483fff640 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c738ae0 0x56536c73af30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1794a0e75f4d5342 server_cookie=6aad3383776ee42e in_seq=0 out_seq=0 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc463fff640 1 -- 172.21.15.28:0/40631402 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56536c688860 con 0x56536c738ae0 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc463fff640 10 client.25277 handle_command_reply: tid=0 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 client.25277 shutdown 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 msgr2=0x56536c72adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc47801b040 tx=0x7fc478004480 comp rx=0 tx=0).stop 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 shutdown_connections 2024-09-19T09:33:22.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c738ae0 0x56536c73af30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c716840 0x56536c728600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c715b80 0x56536c7280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc48d194740 1 --2- 172.21.15.28:0/40631402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c72adc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.515+0000 7fc463fff640 0 client.25277 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.516+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 >> 172.21.15.28:0/40631402 conn(0x56536c6de310 msgr2=0x56536c7179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.516+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 shutdown_connections 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.516+0000 7fc48d194740 1 -- 172.21.15.28:0/40631402 wait complete. 2024-09-19T09:33:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.516+0000 7fc48d194740 20 client.25277 trim_cache size 0 max 16384 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.542+0000 7fc48d194740 1 Processor -- start 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.542+0000 7fc48d194740 1 -- start start 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715b80 0x56536c727060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c710bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:33038/0 (socket says 172.21.15.28:33038) 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc489358640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c710bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc489358640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c710bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53562/0 (socket says 172.21.15.28:53562) 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 -- 172.21.15.28:0/2336967464 learned_addr learned my addr 172.21.15.28:0/2336967464 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841ab480 con 0x56536c716840 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841acfc0 con 0x56536c715120 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841abb50 con 0x56536c715b80 2024-09-19T09:33:22.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc483fff640 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715b80 0x56536c727060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 -- 172.21.15.28:0/2336967464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715b80 msgr2=0x56536c727060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715b80 0x56536c727060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 -- 172.21.15.28:0/2336967464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 msgr2=0x56536c710bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c710bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.543+0000 7fc488b57640 1 -- 172.21.15.28:0/2336967464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56536c7161d0 con 0x56536c715120 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.544+0000 7fc488b57640 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc478009380 tx=0x7fc4780236a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe230a2b647438a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.544+0000 7fc46e7fc640 1 -- 172.21.15.28:0/2336967464 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478027070 con 0x56536c715120 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.544+0000 7fc46e7fc640 1 -- 172.21.15.28:0/2336967464 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478023820 con 0x56536c715120 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.544+0000 7fc46e7fc640 1 -- 172.21.15.28:0/2336967464 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478003d30 con 0x56536c715120 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 msgr2=0x56536c734bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc478009380 tx=0x7fc4780236a0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 shutdown_connections 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c710bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715b80 0x56536c727060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 --2- 172.21.15.28:0/2336967464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715120 0x56536c734bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 >> 172.21.15.28:0/2336967464 conn(0x56536c6de310 msgr2=0x56536c7104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 shutdown_connections 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- 172.21.15.28:0/2336967464 wait complete. 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.545+0000 7fc48d194740 1 -- start start 2024-09-19T09:33:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 Processor -- start 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 0x56536c730ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 0x56536c7313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c72e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc489358640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c72e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc488b57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 0x56536c730ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc489358640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c72e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53576/0 (socket says 172.21.15.28:53576) 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc489358640 1 -- 172.21.15.28:0/608534959 learned_addr learned my addr 172.21.15.28:0/608534959 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841ac8f0 con 0x56536c716840 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841ac220 con 0x56536c715b80 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc488b57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 0x56536c730ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:33122/0 (socket says 172.21.15.28:33122) 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841a9460 con 0x56536c715120 2024-09-19T09:33:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.546+0000 7fc483fff640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 0x56536c7313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc48d194740 10 client.0 _ll_register_callbacks cb 0x7fc4700d2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 -- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 msgr2=0x56536c730ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 0x56536c730ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 -- 172.21.15.28:0/608534959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 msgr2=0x56536c72e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c72e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56536c712960 con 0x56536c715b80 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc483fff640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 0x56536c7313e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4700ccb20 tx=0x7fc470000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bd0bb139a4b401 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc463fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.547+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470009c30 con 0x56536c715b80 2024-09-19T09:33:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc478023820 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fc4700ccd70 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc47009bd30 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc48d194740 10 client.25283 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56536c71c720 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.548+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478023820 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc4700afb50 con 0x56536c715b80 2024-09-19T09:33:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 10 client.25283 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 10 client.25283 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 20 client.25283 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c71ce50 0x56536c71f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 4 client.25283 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56536c682d70 con 0x56536c71ce50 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc488b57640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c71ce50 0x56536c71f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.549+0000 7fc488b57640 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c71ce50 0x56536c71f270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fd95f17d0812409 server_cookie=f0d3aee0efb17934 in_seq=0 out_seq=0 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc463fff640 10 client.25283 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc463fff640 1 -- 172.21.15.28:0/608534959 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56536c682d70 con 0x56536c71ce50 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc463fff640 10 client.25283 handle_command_reply: tid=0 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 client.25283 shutdown 2024-09-19T09:33:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 msgr2=0x56536c7313e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 0x56536c7313e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4700ccb20 tx=0x7fc470000bc0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.557 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 shutdown_connections 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x56536c71ce50 0x56536c71f270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56536c715b80 0x56536c7313e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc463fff640 0 client.25283 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56536c716840 0x56536c72e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 --2- 172.21.15.28:0/608534959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56536c715120 0x56536c730ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 >> 172.21.15.28:0/608534959 conn(0x56536c6de310 msgr2=0x56536c710d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 shutdown_connections 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 1 -- 172.21.15.28:0/608534959 wait complete. 2024-09-19T09:33:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.550+0000 7fc48d194740 20 client.25283 trim_cache size 0 max 16384 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 msgr2=0x7fc4740a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 0x7fc4740a5810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009430 tx=0x7fc47c03f040 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 msgr2=0x7fc484198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d740 tx=0x7fc47000dc00 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 shutdown_connections 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc4740a3350 0x7fc4740a5810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4841082a0 0x7fc48419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4841036e0 0x7fc484198b50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 --2- 172.21.15.28:0/1656505696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc484102d30 0x7fc484198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 >> 172.21.15.28:0/1656505696 conn(0x7fc4840fc7d0 msgr2=0x7fc4840fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.551+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 shutdown_connections 2024-09-19T09:33:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:22.552+0000 7fc48adbb640 1 -- 172.21.15.28:0/1656505696 wait complete. 2024-09-19T09:33:22.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:22 smithi078 ceph-mon[35619]: pgmap v359: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:22.913 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:22 smithi028 ceph-mon[30972]: pgmap v359: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:22.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:22 smithi038 ceph-mon[35271]: pgmap v359: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:22.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b7b0c68d', quiesce took 9.3 seconds. Will release after: 1.8999999999999986 2024-09-19T09:33:23.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:33:23.943 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:24.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 -- 172.21.15.28:0/3558284284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 msgr2=0x7f54a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 --2- 172.21.15.28:0/3558284284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a8101200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5498009900 tx=0x7f549802a480 comp rx=0 tx=0).stop 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 -- 172.21.15.28:0/3558284284 shutdown_connections 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 --2- 172.21.15.28:0/3558284284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 0x7f54a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 --2- 172.21.15.28:0/3558284284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 --2- 172.21.15.28:0/3558284284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a8101200 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 -- 172.21.15.28:0/3558284284 >> 172.21.15.28:0/3558284284 conn(0x7f54a80fc7d0 msgr2=0x7f54a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.467+0000 7f54ae43f640 1 -- 172.21.15.28:0/3558284284 shutdown_connections 2024-09-19T09:33:24.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.468+0000 7f54ae43f640 1 -- 172.21.15.28:0/3558284284 wait complete. 2024-09-19T09:33:24.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.468+0000 7f54ae43f640 1 Processor -- start 2024-09-19T09:33:24.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.468+0000 7f54ae43f640 1 -- start start 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.469+0000 7f54ae43f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a81a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.469+0000 7f54ae43f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.469+0000 7f54ae43f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 0x7f54a810cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.469+0000 7f54ae43f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112060 con 0x7f54a8100e20 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a81a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a81a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53984/0 (socket says 172.21.15.28:53984) 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a7fff640 1 -- 172.21.15.28:0/1302507144 learned_addr learned my addr 172.21.15.28:0/1302507144 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8111ee0 con 0x7f54a8102ef0 2024-09-19T09:33:24.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81121e0 con 0x7f54a8102570 2024-09-19T09:33:24.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 0x7f54a810cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 -- 172.21.15.28:0/1302507144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 msgr2=0x7f54a810cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 0x7f54a810cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 -- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 msgr2=0x7f54a81a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a81a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.470+0000 7f54a77fe640 1 -- 172.21.15.28:0/1302507144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a810d3a0 con 0x7f54a8102570 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54a77fe640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81a4db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f549c00d6d0 tx=0x7f549c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54a57fa640 1 -- 172.21.15.28:0/1302507144 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017070 con 0x7f54a8102570 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a810d690 con 0x7f54a8102570 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81ab6b0 con 0x7f54a8102570 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54a57fa640 1 -- 172.21.15.28:0/1302507144 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549c009820 con 0x7f54a8102570 2024-09-19T09:33:24.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.471+0000 7f54a57fa640 1 -- 172.21.15.28:0/1302507144 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c004cd0 con 0x7f54a8102570 2024-09-19T09:33:24.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.474+0000 7f54a57fa640 1 -- 172.21.15.28:0/1302507144 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f549c005220 con 0x7f54a8102570 2024-09-19T09:33:24.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54a57fa640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 0x7f54900a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 Processor -- start 2024-09-19T09:33:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54a57fa640 1 -- 172.21.15.28:0/1302507144 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f549c0cf370 con 0x7f54a8102570 2024-09-19T09:33:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 -- start start 2024-09-19T09:33:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 0x7f54900a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f22fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fb5d0 0x55d3f22fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d3f22612f0 con 0x55d3f22fcb40 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d3f2208cf0 con 0x55d3f2306370 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.475+0000 7f54b0818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d3f225d2b0 con 0x55d3f22fb5d0 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51106/0 (socket says 172.21.15.28:51106) 2024-09-19T09:33:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 -- 172.21.15.28:0/3631672389 learned_addr learned my addr 172.21.15.28:0/3631672389 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 0x7f54900a5890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5498005770 tx=0x7f5498005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 -- 172.21.15.28:0/3631672389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fb5d0 msgr2=0x55d3f22fba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:33:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fb5d0 0x55d3f22fba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54a7fff640 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f22fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 -- 172.21.15.28:0/3631672389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 msgr2=0x55d3f22fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f22fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54ac9dc640 1 -- 172.21.15.28:0/3631672389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f21ed3a0 con 0x55d3f2306370 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.476+0000 7f54a7fff640 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f22fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.477+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ec0 tx=0x7f54a0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b98dd705b80b0542 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.477+0000 7f548e7fc640 1 -- 172.21.15.28:0/3631672389 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016070 con 0x55d3f2306370 2024-09-19T09:33:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.477+0000 7f548e7fc640 1 -- 172.21.15.28:0/3631672389 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a0004d20 con 0x55d3f2306370 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 -- 172.21.15.28:0/3631672389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 msgr2=0x55d3f2308760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ec0 tx=0x7f54a0010740 comp rx=0 tx=0).stop 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 -- 172.21.15.28:0/3631672389 shutdown_connections 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f2308760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fb5d0 0x55d3f22fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 --2- 172.21.15.28:0/3631672389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f22fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 -- 172.21.15.28:0/3631672389 >> 172.21.15.28:0/3631672389 conn(0x55d3f22c4310 msgr2=0x55d3f22f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.478+0000 7f54b0818740 1 -- 172.21.15.28:0/3631672389 shutdown_connections 2024-09-19T09:33:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.479+0000 7f54b0818740 1 -- 172.21.15.28:0/3631672389 wait complete. 2024-09-19T09:33:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.479+0000 7f54b0818740 1 -- start start 2024-09-19T09:33:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.479+0000 7f54b0818740 1 Processor -- start 2024-09-19T09:33:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f2305d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f2303020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d3f2309110 con 0x55d3f22fcb40 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d3f221c060 con 0x55d3f22fb5d0 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54b0818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d3f2309290 con 0x55d3f2306370 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54ac9dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54ac9dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36962/0 (socket says 172.21.15.28:36962) 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54ac9dc640 1 -- 172.21.15.28:0/1089774700 learned_addr learned my addr 172.21.15.28:0/1089774700 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f2305d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.480+0000 7f54a77fe640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f2303020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54b0818740 10 client.0 _ll_register_callbacks cb 0x55d3f1f6f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 -- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 msgr2=0x55d3f2305d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f2305d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 -- 172.21.15.28:0/1089774700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 msgr2=0x55d3f2303020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f2303020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 -- 172.21.15.28:0/1089774700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f1fd23c0 con 0x55d3f2306370 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54a0006010 tx=0x7f54a0013ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4acb7aa5be9577a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.481+0000 7f54877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0028060 con 0x55d3f2306370 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f54a0003970 con 0x55d3f2306370 2024-09-19T09:33:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d3f226e860 con 0x55d3f2306370 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a0016040 con 0x55d3f2306370 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54b0818740 10 client.25237 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3f2314c90 con 0x55d3f2306370 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.482+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d3f226e860 con 0x55d3f2306370 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54a000f040 con 0x55d3f2306370 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 10 client.25237 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 10 client.25237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 20 client.25237 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f2316450 0x55d3f23188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 4 client.25237 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.483+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f2268d70 con 0x55d3f2316450 2024-09-19T09:33:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.484+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f2316450 0x55d3f23188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.485+0000 7f54877fe640 10 client.25237 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.485+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f2316450 0x55d3f23188b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71580efad3208210 server_cookie=c4f4de843ca938ac in_seq=0 out_seq=0 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.485+0000 7f54877fe640 1 -- 172.21.15.28:0/1089774700 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3f2268d70 con 0x55d3f2316450 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.485+0000 7f54877fe640 10 client.25237 handle_command_reply: tid=0 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.485+0000 7f54b0818740 1 client.25237 shutdown 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 msgr2=0x55d3f2303610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54a0006010 tx=0x7f54a0013ab0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 shutdown_connections 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f2306370 0x55d3f2303610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fcb40 0x55d3f2303020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f2316450 0x55d3f23188b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 --2- 172.21.15.28:0/1089774700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f2305d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 >> 172.21.15.28:0/1089774700 conn(0x55d3f22c4310 msgr2=0x55d3f22fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54877fe640 0 client.25237 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 shutdown_connections 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 1 -- 172.21.15.28:0/1089774700 wait complete. 2024-09-19T09:33:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.486+0000 7f54b0818740 20 client.25237 trim_cache size 0 max 16384 2024-09-19T09:33:24.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.499+0000 7f54b0818740 1 Processor -- start 2024-09-19T09:33:24.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.499+0000 7f54b0818740 1 -- start start 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 0x55d3f2312490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f22f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f22f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f549400e3d0 con 0x55d3f22fb5d0 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f549400ff10 con 0x55d3f2306370 2024-09-19T09:33:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54b0818740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f549400eaa0 con 0x55d3f22fcb40 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f22f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f22f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36976/0 (socket says 172.21.15.28:36976) 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a77fe640 1 -- 172.21.15.28:0/1722893997 learned_addr learned my addr 172.21.15.28:0/1722893997 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.500+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 0x55d3f2312490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f22f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 -- 172.21.15.28:0/1722893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 msgr2=0x55d3f22f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f22f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 -- 172.21.15.28:0/1722893997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 msgr2=0x55d3f22f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f22f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 -- 172.21.15.28:0/1722893997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f22f8180 con 0x55d3f22fb5d0 2024-09-19T09:33:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.501+0000 7f54a7fff640 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 0x55d3f2312490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55d3f2319160 tx=0x7f549803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f651ea281e83f65c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.502+0000 7f548dffb640 1 -- 172.21.15.28:0/1722893997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035040 con 0x55d3f22fb5d0 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.502+0000 7f548dffb640 1 -- 172.21.15.28:0/1722893997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498056070 con 0x55d3f22fb5d0 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.502+0000 7f548dffb640 1 -- 172.21.15.28:0/1722893997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498050310 con 0x55d3f22fb5d0 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 -- 172.21.15.28:0/1722893997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 msgr2=0x55d3f2312490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 0x55d3f2312490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55d3f2319160 tx=0x7f549803f000 comp rx=0 tx=0).stop 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 -- 172.21.15.28:0/1722893997 shutdown_connections 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f2306370 0x55d3f22f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f22f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 --2- 172.21.15.28:0/1722893997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f22fb5d0 0x55d3f2312490 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 -- 172.21.15.28:0/1722893997 >> 172.21.15.28:0/1722893997 conn(0x55d3f22c4310 msgr2=0x55d3f22fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 -- 172.21.15.28:0/1722893997 shutdown_connections 2024-09-19T09:33:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.503+0000 7f54b0818740 1 -- 172.21.15.28:0/1722893997 wait complete. 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.504+0000 7f54b0818740 1 -- start start 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.504+0000 7f54b0818740 1 Processor -- start 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.504+0000 7f54b0818740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f23095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.504+0000 7f54b0818740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f2309b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54b0818740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 0x55d3f230a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f23095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f23095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51128/0 (socket says 172.21.15.28:51128) 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f2309b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54a7fff640 1 -- 172.21.15.28:0/2480387406 learned_addr learned my addr 172.21.15.28:0/2480387406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f549400f840 con 0x55d3f2306370 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 0x55d3f230a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f549400f170 con 0x55d3f22fb5d0 2024-09-19T09:33:24.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f549400c3b0 con 0x55d3f22fcb40 2024-09-19T09:33:24.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.505+0000 7f54ac9dc640 1 -- 172.21.15.28:0/2480387406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 msgr2=0x55d3f2309b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54b0818740 10 client.0 _ll_register_callbacks cb 0x7f54a0009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:24.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f2309b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54ac9dc640 1 -- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 msgr2=0x55d3f23095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:24.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f23095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54ac9dc640 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f237f230 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54ac9dc640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 0x55d3f230a040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54a001a4f0 tx=0x7f54a001a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b7fa0e402576bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5498056070 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54877fe640 10 client.15633 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54877fe640 1 -- 172.21.15.28:0/2480387406 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0033070 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.506+0000 7f54877fe640 1 -- 172.21.15.28:0/2480387406 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f54a00129e0 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54877fe640 1 -- 172.21.15.28:0/2480387406 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a0028070 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d3f23803a0 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54877fe640 1 -- 172.21.15.28:0/2480387406 <== mon.0 v2:172.21.15.28:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54a0036070 con 0x55d3f2306370 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 10 client.15633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 20 client.15633 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f23809b0 0x55d3f2382dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:24.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 4 client.15633 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:33:24.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.507+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3f21e5170 con 0x55d3f23809b0 2024-09-19T09:33:24.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54a7fff640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f23809b0 0x55d3f2382dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54877fe640 10 client.15633 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54a7fff640 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f23809b0 0x55d3f2382dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c56b77eb0f7145ec server_cookie=41d5f51384431c44 in_seq=0 out_seq=0 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54877fe640 1 -- 172.21.15.28:0/2480387406 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3f21e5170 con 0x55d3f23809b0 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54877fe640 10 client.15633 handle_command_reply: tid=0 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.508+0000 7f54b0818740 1 client.15633 shutdown 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 msgr2=0x55d3f230a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 0x55d3f230a040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54a001a4f0 tx=0x7f54a001a990 comp rx=0 tx=0).stop 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 shutdown_connections 2024-09-19T09:33:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d3f2306370 0x55d3f230a040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d3f22fcb40 0x55d3f2309b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x55d3f23809b0 0x55d3f2382dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 --2- 172.21.15.28:0/2480387406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d3f22fb5d0 0x55d3f23095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54877fe640 0 client.15633 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 >> 172.21.15.28:0/2480387406 conn(0x55d3f22c4310 msgr2=0x55d3f22fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 shutdown_connections 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.509+0000 7f54b0818740 1 -- 172.21.15.28:0/2480387406 wait complete. 2024-09-19T09:33:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.510+0000 7f54b0818740 20 client.15633 trim_cache size 0 max 16384 2024-09-19T09:33:24.514 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:33:24.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.510+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 msgr2=0x7f54900a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.510+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 0x7f54900a5890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5498005770 tx=0x7f5498005a20 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.510+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 msgr2=0x7f54a81a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.510+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81a4db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f549c00d6d0 tx=0x7f549c00db90 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 shutdown_connections 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f54900a33d0 0x7f54900a5890 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a8102ef0 0x7f54a810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a8102570 0x7f54a81a4db0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 --2- 172.21.15.28:0/1302507144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a8100e20 0x7f54a81a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 >> 172.21.15.28:0/1302507144 conn(0x7f54a80fc7d0 msgr2=0x7f54a8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 shutdown_connections 2024-09-19T09:33:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:24.511+0000 7f54ae43f640 1 -- 172.21.15.28:0/1302507144 wait complete. 2024-09-19T09:33:24.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:24 smithi028 ceph-mon[30972]: pgmap v360: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:24.852 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:33:24.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:24 smithi078 ceph-mon[35619]: pgmap v360: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:24.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:24 smithi038 ceph-mon[35271]: pgmap v360: 97 pgs: 97 active+clean; 668 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:33:25.232 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:33:25.315 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:25.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 -- 172.21.15.28:0/2726613853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 msgr2=0x7efe5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:25.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 --2- 172.21.15.28:0/2726613853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1029b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009900 tx=0x7efe4c02a480 comp rx=0 tx=0).stop 2024-09-19T09:33:25.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 -- 172.21.15.28:0/2726613853 shutdown_connections 2024-09-19T09:33:25.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 --2- 172.21.15.28:0/2726613853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe5c102ef0 0x7efe5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 --2- 172.21.15.28:0/2726613853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1029b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 --2- 172.21.15.28:0/2726613853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 0x7efe5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 -- 172.21.15.28:0/2726613853 >> 172.21.15.28:0/2726613853 conn(0x7efe5c0fc7d0 msgr2=0x7efe5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 -- 172.21.15.28:0/2726613853 shutdown_connections 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.836+0000 7efe645a7640 1 -- 172.21.15.28:0/2726613853 wait complete. 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.837+0000 7efe645a7640 1 Processor -- start 2024-09-19T09:33:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.837+0000 7efe645a7640 1 -- start start 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.837+0000 7efe645a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 0x7efe5c10d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe645a7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe645a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe5c102ef0 0x7efe5c109de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe645a7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c110f80 con 0x7efe5c102570 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe62343640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 0x7efe5c10d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54050/0 (socket says 172.21.15.28:54050) 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 -- 172.21.15.28:0/685114836 learned_addr learned my addr 172.21.15.28:0/685114836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c110e00 con 0x7efe5c102ef0 2024-09-19T09:33:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c111100 con 0x7efe5c100e20 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 -- 172.21.15.28:0/685114836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 msgr2=0x7efe5c10d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 0x7efe5c10d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 -- 172.21.15.28:0/685114836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe5c102ef0 msgr2=0x7efe5c109de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe5c102ef0 0x7efe5c109de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c10a6a0 con 0x7efe5c102570 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.838+0000 7efe61b42640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efe4c006010 tx=0x7efe4c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.839+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c03f070 con 0x7efe5c102570 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.839+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1ab0e0 con 0x7efe5c102570 2024-09-19T09:33:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.839+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c031c30 con 0x7efe5c102570 2024-09-19T09:33:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.839+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c035030 con 0x7efe5c102570 2024-09-19T09:33:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.839+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1ab640 con 0x7efe5c102570 2024-09-19T09:33:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.840+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c1021b0 con 0x7efe5c102570 2024-09-19T09:33:25.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.843+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7efe4c042070 con 0x7efe5c102570 2024-09-19T09:33:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.844+0000 7efe537fe640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 0x7efe480a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.844+0000 7efe62343640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 0x7efe480a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.844+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efe4c0efe70 con 0x7efe5c102570 2024-09-19T09:33:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.845+0000 7efe62343640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 0x7efe480a5890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe58009bd0 tx=0x7efe58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:25.852+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe4c0bca00 con 0x7efe5c102570 2024-09-19T09:33:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.157+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5c1ab960 con 0x7efe5c102570 2024-09-19T09:33:26.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.159+0000 7efe537fe640 1 -- 172.21.15.28:0/685114836 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7efe4c0bc300 con 0x7efe5c102570 2024-09-19T09:33:26.161 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:33:26.161 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:33:26.161 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.162+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 msgr2=0x7efe480a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.162+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 0x7efe480a5890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe58009bd0 tx=0x7efe58005f90 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.162+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 msgr2=0x7efe5c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.162+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efe4c006010 tx=0x7efe4c003c10 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 shutdown_connections 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efe480a33d0 0x7efe480a5890 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe5c102ef0 0x7efe5c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe5c102570 0x7efe5c1098a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 --2- 172.21.15.28:0/685114836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe5c100e20 0x7efe5c10d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 >> 172.21.15.28:0/685114836 conn(0x7efe5c0fc7d0 msgr2=0x7efe5c103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 shutdown_connections 2024-09-19T09:33:26.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.163+0000 7efe645a7640 1 -- 172.21.15.28:0/685114836 wait complete. 2024-09-19T09:33:26.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 -- 172.21.15.28:0/3571419652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1082a0 msgr2=0x7fabfc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 --2- 172.21.15.28:0/3571419652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1082a0 0x7fabfc10ec40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009cd0 tx=0x7fabf402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 -- 172.21.15.28:0/3571419652 shutdown_connections 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 --2- 172.21.15.28:0/3571419652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1036e0 0x7fabfc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 --2- 172.21.15.28:0/3571419652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1082a0 0x7fabfc10ec40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 --2- 172.21.15.28:0/3571419652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 -- 172.21.15.28:0/3571419652 >> 172.21.15.28:0/3571419652 conn(0x7fabfc0fc7d0 msgr2=0x7fabfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 -- 172.21.15.28:0/3571419652 shutdown_connections 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.382+0000 7fac01840640 1 -- 172.21.15.28:0/3571419652 wait complete. 2024-09-19T09:33:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 Processor -- start 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 -- start start 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 0x7fabfc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 0x7fabfc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10ff00 con 0x7fabfc1036e0 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10fd80 con 0x7fabfc1082a0 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.383+0000 7fac01840640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc110080 con 0x7fabfc102d30 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 0x7fabfc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 0x7fabfc196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54068/0 (socket says 172.21.15.28:54068) 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfa7fc640 1 -- 172.21.15.28:0/1189483018 learned_addr learned my addr 172.21.15.28:0/1189483018 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 0x7fabfc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfaffd640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 -- 172.21.15.28:0/1189483018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 msgr2=0x7fabfc196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 -- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 msgr2=0x7fabfc196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 0x7fabfc196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 -- 172.21.15.28:0/1189483018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc19b2f0 con 0x7fabfc1082a0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfaffd640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.384+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 0x7fabfc19ac10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009ca0 tx=0x7fabf4031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.385+0000 7fac0083e640 1 -- 172.21.15.28:0/1189483018 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4004040 con 0x7fabfc1082a0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.385+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc19b580 con 0x7fabfc1082a0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.385+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a2ec0 con 0x7fabfc1082a0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.385+0000 7fac0083e640 1 -- 172.21.15.28:0/1189483018 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf40046c0 con 0x7fabfc1082a0 2024-09-19T09:33:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.385+0000 7fac0083e640 1 -- 172.21.15.28:0/1189483018 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4030e40 con 0x7fabfc1082a0 2024-09-19T09:33:26.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.387+0000 7fac03c19740 1 Processor -- start 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac0083e640 1 -- 172.21.15.28:0/1189483018 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fabf4046480 con 0x7fabfc1082a0 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 -- start start 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 0x5570fdba0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdbad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5570fdb062f0 con 0x5570fdba1b40 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5570fdaadcf0 con 0x5570fdbab370 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.388+0000 7fac03c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570fdb022b0 con 0x5570fdba05d0 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37030/0 (socket says 172.21.15.28:37030) 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 -- 172.21.15.28:0/2890164944 learned_addr learned my addr 172.21.15.28:0/2890164944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 0x5570fdba0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 -- 172.21.15.28:0/2890164944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 msgr2=0x5570fdbad760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdbad760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 -- 172.21.15.28:0/2890164944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 msgr2=0x5570fdba0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 0x5570fdba0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 -- 172.21.15.28:0/2890164944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fda923a0 con 0x5570fdba05d0 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 0x5570fdba0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fac0083e640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 0x7fabe40a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.389+0000 7fabfaffd640 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabec015670 tx=0x7fabec005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b089041fa4ba11e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.390+0000 7fac0083e640 1 -- 172.21.15.28:0/1189483018 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fabf402e030 con 0x7fabfc1082a0 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.390+0000 7fabe1ffb640 1 -- 172.21.15.28:0/2890164944 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec01d070 con 0x5570fdba05d0 2024-09-19T09:33:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.390+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 0x7fabe40a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.390+0000 7fabe1ffb640 1 -- 172.21.15.28:0/2890164944 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabec00bab0 con 0x5570fdba05d0 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.390+0000 7fabe1ffb640 1 -- 172.21.15.28:0/2890164944 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec013c30 con 0x5570fdba05d0 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 -- 172.21.15.28:0/2890164944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 msgr2=0x5570fdba0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 0x7fabe40a5890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5570fdba1070 tx=0x7fabf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabec015670 tx=0x7fabec005f90 comp rx=0 tx=0).stop 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 -- 172.21.15.28:0/2890164944 shutdown_connections 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdbad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdba0a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 --2- 172.21.15.28:0/2890164944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba1b40 0x5570fdba0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 -- 172.21.15.28:0/2890164944 >> 172.21.15.28:0/2890164944 conn(0x5570fdb69310 msgr2=0x5570fdb9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 -- 172.21.15.28:0/2890164944 shutdown_connections 2024-09-19T09:33:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.391+0000 7fac03c19740 1 -- 172.21.15.28:0/2890164944 wait complete. 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 -- start start 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 Processor -- start 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbaac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdba8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5570fdbae070 con 0x5570fdba05d0 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5570fdac1060 con 0x5570fdbab370 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570fdbae1f0 con 0x5570fdba1b40 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.392+0000 7fac03c19740 10 client.0 _ll_register_callbacks cb 0x5570fd8148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37040/0 (socket says 172.21.15.28:37040) 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 -- 172.21.15.28:0/802794530 learned_addr learned my addr 172.21.15.28:0/802794530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdba8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfaffd640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbaac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 -- 172.21.15.28:0/802794530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 msgr2=0x5570fdba8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdba8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 -- 172.21.15.28:0/802794530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 msgr2=0x5570fdbaac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbaac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.393+0000 7fabfa7fc640 1 -- 172.21.15.28:0/802794530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fd8773c0 con 0x5570fdba1b40 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011d30 tx=0x7fabf00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b81deaec07df7644 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004030 con 0x5570fdba1b40 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fabf00041d0 con 0x5570fdba1b40 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabec00bab0 con 0x5570fdba1b40 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabf00043d0 con 0x5570fdba1b40 2024-09-19T09:33:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fac03c19740 10 client.25261 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570fdbb9a30 con 0x5570fdba1b40 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.394+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabec00bab0 con 0x5570fdba1b40 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabf001a040 con 0x5570fdba1b40 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 10 client.25261 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 10 client.25261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 20 client.25261 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdbbb410 0x5570fdbbd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 4 client.25261 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570fdb13860 con 0x5570fdbbb410 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.395+0000 7fabfaffd640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdbbb410 0x5570fdbbd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.396+0000 7fabdaffd640 10 client.25261 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.396+0000 7fabfaffd640 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdbbb410 0x5570fdbbd8b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0de3c5e395fbdd server_cookie=e394c5c2ea376d4c in_seq=0 out_seq=0 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fabdaffd640 1 -- 172.21.15.28:0/802794530 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570fdb13860 con 0x5570fdbbb410 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fabdaffd640 10 client.25261 handle_command_reply: tid=0 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 client.25261 shutdown 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 msgr2=0x5570fdba7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011d30 tx=0x7fabf00046c0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 shutdown_connections 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdbab370 0x5570fdba8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdbbb410 0x5570fdbbd8b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba1b40 0x5570fdba7ec0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 --2- 172.21.15.28:0/802794530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbaac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 >> 172.21.15.28:0/802794530 conn(0x5570fdb69310 msgr2=0x5570fdba2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fabdaffd640 0 client.25261 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 shutdown_connections 2024-09-19T09:33:26.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 1 -- 172.21.15.28:0/802794530 wait complete. 2024-09-19T09:33:26.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.397+0000 7fac03c19740 20 client.25261 trim_cache size 0 max 16384 2024-09-19T09:33:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 Processor -- start 2024-09-19T09:33:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 -- start start 2024-09-19T09:33:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdbb73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdb9ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 0x5570fdbc3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800e380 con 0x5570fdbab370 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800fec0 con 0x5570fdba1b40 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.414+0000 7fac03c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800ea50 con 0x5570fdba05d0 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 0x5570fdbc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 0x5570fdbc3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54084/0 (socket says 172.21.15.28:54084) 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfb7fe640 1 -- 172.21.15.28:0/446296522 learned_addr learned my addr 172.21.15.28:0/446296522 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdb9ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfaffd640 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdbb73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 -- 172.21.15.28:0/446296522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 msgr2=0x5570fdbb73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdbb73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 -- 172.21.15.28:0/446296522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 msgr2=0x5570fdbc3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 0x5570fdbc3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 -- 172.21.15.28:0/446296522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fdba1530 con 0x5570fdba1b40 2024-09-19T09:33:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.415+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdb9ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fabf0018040 tx=0x7fabf0011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09f17d997348771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.416+0000 7fabe17fa640 1 -- 172.21.15.28:0/446296522 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004930 con 0x5570fdba1b40 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.416+0000 7fabe17fa640 1 -- 172.21.15.28:0/446296522 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0035070 con 0x5570fdba1b40 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.416+0000 7fabe17fa640 1 -- 172.21.15.28:0/446296522 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf001c8f0 con 0x5570fdba1b40 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- 172.21.15.28:0/446296522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 msgr2=0x5570fdb9ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdb9ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fabf0018040 tx=0x7fabf0011f70 comp rx=0 tx=0).stop 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- 172.21.15.28:0/446296522 shutdown_connections 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdb9ce10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdbab370 0x5570fdbc3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 --2- 172.21.15.28:0/446296522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdba05d0 0x5570fdbb73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- 172.21.15.28:0/446296522 >> 172.21.15.28:0/446296522 conn(0x5570fdb69310 msgr2=0x5570fdb9c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- 172.21.15.28:0/446296522 shutdown_connections 2024-09-19T09:33:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- 172.21.15.28:0/446296522 wait complete. 2024-09-19T09:33:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.417+0000 7fac03c19740 1 -- start start 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 Processor -- start 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdbb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800f7f0 con 0x5570fdba05d0 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800f120 con 0x5570fdba1b40 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800c360 con 0x5570fdbab370 2024-09-19T09:33:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fac03c19740 10 client.0 _ll_register_callbacks cb 0x7fabf00119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54098/0 (socket says 172.21.15.28:54098) 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37068/0 (socket says 172.21.15.28:37068) 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfaffd640 1 -- 172.21.15.28:0/3317276869 learned_addr learned my addr 172.21.15.28:0/3317276869 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.418+0000 7fabfa7fc640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdbb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 -- 172.21.15.28:0/3317276869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 msgr2=0x5570fdbb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdbb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 -- 172.21.15.28:0/3317276869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 msgr2=0x5570fdbb36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 -- 172.21.15.28:0/3317276869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fdba4e20 con 0x5570fdbab370 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfaffd640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabfb7fe640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabf40c5a40 tx=0x7fabf40ee7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ae8d020c9e8882 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40d5020 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fabf40ee930 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf0035070 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabf40bda90 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fac03c19740 10 client.25264 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.419+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570fdc2e2e0 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0035070 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabf404e070 con 0x5570fdbab370 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 10 client.25264 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 20 client.25264 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdc2e9b0 0x5570fdc30dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 4 client.25264 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570fdb0dd70 con 0x5570fdc2e9b0 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fabfaffd640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdc2e9b0 0x5570fdc30dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.420+0000 7fabdaffd640 10 client.25264 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fabfaffd640 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdc2e9b0 0x5570fdc30dd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5aafda6e356e83 server_cookie=ebec2cdc8d3d915f in_seq=0 out_seq=0 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fabdaffd640 1 -- 172.21.15.28:0/3317276869 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570fdb0dd70 con 0x5570fdc2e9b0 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fabdaffd640 10 client.25264 handle_command_reply: tid=0 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 client.25264 shutdown 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 msgr2=0x5570fdba4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabf40c5a40 tx=0x7fabf40ee7c0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 shutdown_connections 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5570fdba1b40 0x5570fdbb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5570fdc2e9b0 0x5570fdc30dd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fabdaffd640 0 client.25264 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570fdbab370 0x5570fdba4730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 --2- 172.21.15.28:0/3317276869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5570fdba05d0 0x5570fdbb36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 >> 172.21.15.28:0/3317276869 conn(0x5570fdb69310 msgr2=0x5570fdba9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 shutdown_connections 2024-09-19T09:33:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 1 -- 172.21.15.28:0/3317276869 wait complete. 2024-09-19T09:33:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.421+0000 7fac03c19740 20 client.25264 trim_cache size 0 max 16384 2024-09-19T09:33:26.428 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 msgr2=0x7fabe40a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 0x7fabe40a5890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5570fdba1070 tx=0x7fabf0005f90 comp rx=0 tx=0).stop 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 msgr2=0x7fabfc19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 0x7fabfc19ac10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009ca0 tx=0x7fabf4031e60 comp rx=0 tx=0).stop 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 shutdown_connections 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fabe40a33d0 0x7fabe40a5890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fabfc1082a0 0x7fabfc19ac10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fabfc1036e0 0x7fabfc196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 --2- 172.21.15.28:0/1189483018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabfc102d30 0x7fabfc196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.422+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 >> 172.21.15.28:0/1189483018 conn(0x7fabfc0fc7d0 msgr2=0x7fabfc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.423+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 shutdown_connections 2024-09-19T09:33:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:26.423+0000 7fac01840640 1 -- 172.21.15.28:0/1189483018 wait complete. 2024-09-19T09:33:26.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:26 smithi078 ceph-mon[35619]: pgmap v361: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:33:26.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:26 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/685114836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:26 smithi038 ceph-mon[35271]: pgmap v361: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:33:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:26 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/685114836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:27.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:26 smithi028 ceph-mon[30972]: pgmap v361: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:33:27.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:26 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/685114836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:27.126 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id b7b0c68d --release --await' 2024-09-19T09:33:27.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id b7b0c68d --release --await 2024-09-19T09:33:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.464+0000 7f944d104640 1 -- 172.21.15.28:0/516531308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 msgr2=0x7f9448103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.464+0000 7f944d104640 1 --2- 172.21.15.28:0/516531308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448103110 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9438009900 tx=0x7f943802a480 comp rx=0 tx=0).stop 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 -- 172.21.15.28:0/516531308 shutdown_connections 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 --2- 172.21.15.28:0/516531308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 --2- 172.21.15.28:0/516531308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f944810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 --2- 172.21.15.28:0/516531308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448103110 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 -- 172.21.15.28:0/516531308 >> 172.21.15.28:0/516531308 conn(0x7f94480fc790 msgr2=0x7f94480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 -- 172.21.15.28:0/516531308 shutdown_connections 2024-09-19T09:33:28.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 -- 172.21.15.28:0/516531308 wait complete. 2024-09-19T09:33:28.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.465+0000 7f944d104640 1 Processor -- start 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944d104640 1 -- start start 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944d104640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944d104640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944d104640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37096/0 (socket says 172.21.15.28:37096) 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f9446d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944659c640 1 -- 172.21.15.28:0/1871596460 learned_addr learned my addr 172.21.15.28:0/1871596460 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9448111f60 con 0x7f9448102d30 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.466+0000 7f944759e640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9448111de0 con 0x7f944810c160 2024-09-19T09:33:28.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94481120e0 con 0x7f9448103650 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f9446d9d640 1 -- 172.21.15.28:0/1871596460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 msgr2=0x7f9448198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f9446d9d640 1 -- 172.21.15.28:0/1871596460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 msgr2=0x7f944819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f9446d9d640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944819d3e0 con 0x7f9448102d30 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f944759e640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.467+0000 7f944659c640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448198430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9438005940 tx=0x7f94380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f942ffff640 1 -- 172.21.15.28:0/1871596460 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943803f030 con 0x7f9448102d30 2024-09-19T09:33:28.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f942ffff640 1 -- 172.21.15.28:0/1871596460 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9438003ce0 con 0x7f9448102d30 2024-09-19T09:33:28.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944819d670 con 0x7f9448102d30 2024-09-19T09:33:28.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481a4fb0 con 0x7f9448102d30 2024-09-19T09:33:28.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.468+0000 7f942ffff640 1 -- 172.21.15.28:0/1871596460 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438035070 con 0x7f9448102d30 2024-09-19T09:33:28.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 Processor -- start 2024-09-19T09:33:28.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 -- start start 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89124840 0x564b89133110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89133650 0x564b89135a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564b89089620 con 0x564b89123f20 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564b89030e50 con 0x564b89133650 2024-09-19T09:33:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944f4dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b890855e0 con 0x564b89124840 2024-09-19T09:33:28.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944759e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54116/0 (socket says 172.21.15.28:54116) 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.471+0000 7f944759e640 1 -- 172.21.15.28:0/3620180422 learned_addr learned my addr 172.21.15.28:0/3620180422 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944659c640 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89133650 0x564b89135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 -- 172.21.15.28:0/3620180422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89124840 msgr2=0x564b89133110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89124840 0x564b89133110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 -- 172.21.15.28:0/3620180422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89133650 msgr2=0x564b89135a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89133650 0x564b89135a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 -- 172.21.15.28:0/3620180422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b89021d40 con 0x564b89123f20 2024-09-19T09:33:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.472+0000 7f944759e640 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9440009f30 tx=0x7f9440010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df46b663b9de61a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.473+0000 7f942d7fa640 1 -- 172.21.15.28:0/3620180422 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440016070 con 0x564b89123f20 2024-09-19T09:33:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.473+0000 7f942d7fa640 1 -- 172.21.15.28:0/3620180422 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944001bb40 con 0x564b89123f20 2024-09-19T09:33:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.473+0000 7f942d7fa640 1 -- 172.21.15.28:0/3620180422 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440012c90 con 0x564b89123f20 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.473+0000 7f942ffff640 1 -- 172.21.15.28:0/1871596460 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f9438042030 con 0x7f9448102d30 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 -- 172.21.15.28:0/3620180422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 msgr2=0x564b89124300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9440009f30 tx=0x7f9440010800 comp rx=0 tx=0).stop 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 -- 172.21.15.28:0/3620180422 shutdown_connections 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89133650 0x564b89135a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89124840 0x564b89133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3620180422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89124300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f944f4dd740 1 -- 172.21.15.28:0/3620180422 >> 172.21.15.28:0/3620180422 conn(0x564b891205f0 msgr2=0x564b891209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.474+0000 7f942ffff640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 0x7f94300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944f4dd740 1 -- 172.21.15.28:0/3620180422 shutdown_connections 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f942ffff640 1 -- 172.21.15.28:0/1871596460 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94380efa10 con 0x7f9448102d30 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944659c640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 0x7f94300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944f4dd740 1 -- 172.21.15.28:0/3620180422 wait complete. 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944659c640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 0x7f94300a57b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x564b891216e0 tx=0x7f943c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944f4dd740 1 -- start start 2024-09-19T09:33:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.475+0000 7f944f4dd740 1 Processor -- start 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b89130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 0x564b891310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564b89136220 con 0x564b89123f20 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564b89011f10 con 0x564b89124840 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b891363a0 con 0x564b89133650 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944f4dd740 10 client.0 _ll_register_callbacks cb 0x564b89035db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b89130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b89130b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51248/0 (socket says 172.21.15.28:51248) 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944659c640 1 -- 172.21.15.28:0/3416618242 learned_addr learned my addr 172.21.15.28:0/3416618242 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f9446d9d640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.476+0000 7f944759e640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 0x564b891310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 -- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 msgr2=0x564b891310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 0x564b891310b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 -- 172.21.15.28:0/3416618242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 msgr2=0x564b89130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b89130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 -- 172.21.15.28:0/3416618242 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b88d98580 con 0x564b89123f20 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f944759e640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 0x564b891310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f942aff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f9446d9d640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89130630 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94380ed390 tx=0x7f94380bbc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c421fbb2188f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.477+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438049070 con 0x564b89123f20 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944001bb40 con 0x564b89123f20 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94380be020 con 0x564b89123f20 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9438049070 con 0x564b89123f20 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 10 client.15666 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b88e62960 con 0x564b89123f20 2024-09-19T09:33:28.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944001bb40 con 0x564b89123f20 2024-09-19T09:33:28.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f943804c060 con 0x564b89123f20 2024-09-19T09:33:28.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:28.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 10 client.15666 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 20 client.15666 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b8912a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 4 client.15666 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.478+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b89096b90 con 0x564b891281f0 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.479+0000 7f944659c640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b8912a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.479+0000 7f944659c640 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b8912a610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e56487d8c09df9b server_cookie=b12b8f75010319be in_seq=0 out_seq=0 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f942aff5640 10 client.15666 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f942aff5640 1 -- 172.21.15.28:0/3416618242 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b89096b90 con 0x564b891281f0 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f942aff5640 10 client.15666 handle_command_reply: tid=0 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 client.15666 shutdown 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 msgr2=0x564b89130630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89130630 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94380ed390 tx=0x7f94380bbc70 comp rx=0 tx=0).stop 2024-09-19T09:33:28.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 shutdown_connections 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b89130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.480+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b8912a610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b89133650 0x564b891310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 1 --2- 172.21.15.28:0/3416618242 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b89130630 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f942aff5640 0 client.15666 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 >> 172.21.15.28:0/3416618242 conn(0x564b891205f0 msgr2=0x564b89126350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 shutdown_connections 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 1 -- 172.21.15.28:0/3416618242 wait complete. 2024-09-19T09:33:28.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.481+0000 7f944f4dd740 20 client.15666 trim_cache size 0 max 16384 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.514+0000 7f944f4dd740 1 Processor -- start 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.514+0000 7f944f4dd740 1 -- start start 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b8913edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b8912ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f943400e2e0 con 0x564b89123f20 2024-09-19T09:33:28.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f943400fe20 con 0x564b89124840 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944f4dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f943400e9b0 con 0x564b891205f0 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54124/0 (socket says 172.21.15.28:54124) 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 -- 172.21.15.28:0/1066384858 learned_addr learned my addr 172.21.15.28:0/1066384858 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944759e640 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b8912ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b8913edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 -- 172.21.15.28:0/1066384858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 msgr2=0x564b8913edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b8913edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 -- 172.21.15.28:0/1066384858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 msgr2=0x564b8912ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b8912ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.515+0000 7f944659c640 1 -- 172.21.15.28:0/1066384858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b89121710 con 0x564b89123f20 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.516+0000 7f944659c640 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c5c2b065cc5dac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.516+0000 7f942cff9640 1 -- 172.21.15.28:0/1066384858 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c01d9a0 con 0x564b89123f20 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.516+0000 7f942cff9640 1 -- 172.21.15.28:0/1066384858 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c01db40 con 0x564b89123f20 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.516+0000 7f942cff9640 1 -- 172.21.15.28:0/1066384858 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0296d0 con 0x564b89123f20 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- 172.21.15.28:0/1066384858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 msgr2=0x564b8912d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c015000 comp rx=0 tx=0).stop 2024-09-19T09:33:28.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- 172.21.15.28:0/1066384858 shutdown_connections 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89124840 0x564b8912ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89123f20 0x564b8912d890 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1066384858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b8913edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- 172.21.15.28:0/1066384858 >> 172.21.15.28:0/1066384858 conn(0x564b89131210 msgr2=0x564b8913e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- 172.21.15.28:0/1066384858 shutdown_connections 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- 172.21.15.28:0/1066384858 wait complete. 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 -- start start 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.517+0000 7f944f4dd740 1 Processor -- start 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89123f20 0x564b89145c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89124840 0x564b891a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f943400f750 con 0x564b89124840 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f943400f080 con 0x564b89123f20 2024-09-19T09:33:28.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f943400c2c0 con 0x564b891205f0 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f9446d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f9446d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37114/0 (socket says 172.21.15.28:37114) 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f9446d9d640 1 -- 172.21.15.28:0/1362621857 learned_addr learned my addr 172.21.15.28:0/1362621857 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944759e640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89124840 0x564b891a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944659c640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89123f20 0x564b89145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.518+0000 7f944f4dd740 10 client.0 _ll_register_callbacks cb 0x7f9438009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:28.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 -- 172.21.15.28:0/1362621857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89123f20 msgr2=0x564b89145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89123f20 0x564b89145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 -- 172.21.15.28:0/1362621857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89124840 msgr2=0x564b891a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89124840 0x564b891a30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 -- 172.21.15.28:0/1362621857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b891a3710 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f9446d9d640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94380bbba0 tx=0x7f9438000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b9e8a891cde36f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f942aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380bb9e0 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94380be070 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c01db40 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94380bc5a0 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f944f4dd740 10 client.25276 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.519+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b891aaac0 con 0x564b891205f0 2024-09-19T09:33:28.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94380bbea0 con 0x564b891205f0 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f943804c060 con 0x564b891205f0 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 10 client.25276 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 10 client.25276 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 20 client.25276 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b891acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 4 client.25276 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b7b0c68d", "release": true, "await": true}] 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b7b0c68d", "release": true, "await": true}) -- 0x7f943c01db40 con 0x564b891281f0 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944659c640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b891acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.520+0000 7f944659c640 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b891acfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92851d74788f831 server_cookie=143b4f5732bf0c33 in_seq=0 out_seq=0 2024-09-19T09:33:28.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.521+0000 7f942aff5640 10 client.25276 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f942aff5640 1 -- 172.21.15.28:0/1362621857 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f943c01db40 con 0x564b891281f0 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f942aff5640 10 client.25276 handle_command_reply: tid=0 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 client.25276 shutdown 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 msgr2=0x564b891456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94380bbba0 tx=0x7f9438000bc0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 shutdown_connections 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564b89123f20 0x564b89145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564b891281f0 0x564b891acfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f942aff5640 0 client.25276 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564b89124840 0x564b891a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 --2- 172.21.15.28:0/1362621857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b891205f0 0x564b891456c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 >> 172.21.15.28:0/1362621857 conn(0x564b89131210 msgr2=0x564b8913eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.523+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 shutdown_connections 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944f4dd740 1 -- 172.21.15.28:0/1362621857 wait complete. 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944f4dd740 20 client.25276 trim_cache size 0 max 16384 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 4, 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:33:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: "b7b0c68d": { 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 4, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 131.2, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:28.529 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:28.530 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 msgr2=0x7f94300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 0x7f94300a57b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x564b891216e0 tx=0x7f943c00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 msgr2=0x7f9448198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.524+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448198430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9438005940 tx=0x7f94380042f0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 shutdown_connections 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94300a32f0 0x7f94300a57b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f944810c160 0x7f944819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9448103650 0x7f9448198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 --2- 172.21.15.28:0/1871596460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9448102d30 0x7f9448198430 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 >> 172.21.15.28:0/1871596460 conn(0x7f94480fc790 msgr2=0x7f94480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 shutdown_connections 2024-09-19T09:33:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:28.525+0000 7f944d104640 1 -- 172.21.15.28:0/1871596460 wait complete. 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- 172.21.15.28:0/4196735548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 msgr2=0x7f53041020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 --2- 172.21.15.28:0/4196735548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f53041020b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009900 tx=0x7f52f402a410 comp rx=0 tx=0).stop 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- 172.21.15.28:0/4196735548 shutdown_connections 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 --2- 172.21.15.28:0/4196735548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 0x7f5304108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 --2- 172.21.15.28:0/4196735548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f53041020b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 --2- 172.21.15.28:0/4196735548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f5304100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- 172.21.15.28:0/4196735548 >> 172.21.15.28:0/4196735548 conn(0x7f53040fc1c0 msgr2=0x7f53040fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- 172.21.15.28:0/4196735548 shutdown_connections 2024-09-19T09:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- 172.21.15.28:0/4196735548 wait complete. 2024-09-19T09:33:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 Processor -- start 2024-09-19T09:33:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.616+0000 7f530c1c0640 1 -- start start 2024-09-19T09:33:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f53041049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 0x7f5304103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f530410c6f0 con 0x7f5304101c30 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f530410c570 con 0x7f53041025f0 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530c1c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f530410c870 con 0x7f5304100a40 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54142/0 (socket says 172.21.15.28:54142) 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 -- 172.21.15.28:0/1989293884 learned_addr learned my addr 172.21.15.28:0/1989293884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530a75d640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 0x7f5304103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f53041049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 -- 172.21.15.28:0/1989293884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 msgr2=0x7f53041049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f53041049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 -- 172.21.15.28:0/1989293884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 msgr2=0x7f5304103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 0x7f5304103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 -- 172.21.15.28:0/1989293884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5304103de0 con 0x7f5304101c30 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f53041049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.617+0000 7f530975b640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f52f4031440 tx=0x7f52f4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.618+0000 7f52faffd640 1 -- 172.21.15.28:0/1989293884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f403f030 con 0x7f5304101c30 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.618+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a1f50 con 0x7f5304101c30 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.618+0000 7f52faffd640 1 -- 172.21.15.28:0/1989293884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f4003ce0 con 0x7f5304101c30 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.618+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a23c0 con 0x7f5304101c30 2024-09-19T09:33:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.618+0000 7f52faffd640 1 -- 172.21.15.28:0/1989293884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4035070 con 0x7f5304101c30 2024-09-19T09:33:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.621+0000 7f52faffd640 1 -- 172.21.15.28:0/1989293884 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f52f4042030 con 0x7f5304101c30 2024-09-19T09:33:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.621+0000 7f52faffd640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 0x7f52f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.621+0000 7f52faffd640 1 -- 172.21.15.28:0/1989293884 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f52f40f0000 con 0x7f5304101c30 2024-09-19T09:33:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.621+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 0x7f52f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.622+0000 7f530f02f740 1 Processor -- start 2024-09-19T09:33:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.622+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 0x7f52f00a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53000058c0 tx=0x7f5300005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.622+0000 7f530f02f740 1 -- start start 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d521dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d5225830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5558d51812f0 con 0x5558d521e2e0 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54144/0 (socket says 172.21.15.28:54144) 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530975b640 1 -- 172.21.15.28:0/2096071096 learned_addr learned my addr 172.21.15.28:0/2096071096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5558d5129270 con 0x5558d5223370 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d521dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5558d517d2b0 con 0x5558d521d9a0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.623+0000 7f530a75d640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d5225830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 -- 172.21.15.28:0/2096071096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 msgr2=0x5558d521dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d521dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 -- 172.21.15.28:0/2096071096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 msgr2=0x5558d5225830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d5225830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 -- 172.21.15.28:0/2096071096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d5128340 con 0x5558d521e2e0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530a75d640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d5225830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f530975b640 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52f40c4c30 tx=0x7f52f4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2eaf67e94240ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f52effff640 1 -- 172.21.15.28:0/2096071096 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40bc4f0 con 0x5558d521e2e0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f52effff640 1 -- 172.21.15.28:0/2096071096 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f40c08a0 con 0x5558d521e2e0 2024-09-19T09:33:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.624+0000 7f52effff640 1 -- 172.21.15.28:0/2096071096 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f403b460 con 0x5558d521e2e0 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 msgr2=0x5558d5222da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f52f40c4c30 tx=0x7f52f4005e30 comp rx=0 tx=0).stop 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 shutdown_connections 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d5225830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5222da0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 --2- 172.21.15.28:0/2096071096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d521dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 >> 172.21.15.28:0/2096071096 conn(0x5558d5219010 msgr2=0x5558d5219410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 shutdown_connections 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.625+0000 7f530f02f740 1 -- 172.21.15.28:0/2096071096 wait complete. 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 -- start start 2024-09-19T09:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 Processor -- start 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d5227890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d522ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5558d5226100 con 0x5558d521e2e0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5558d510a290 con 0x5558d5223370 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5558d5226280 con 0x5558d521d9a0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54152/0 (socket says 172.21.15.28:54152) 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530975b640 1 -- 172.21.15.28:0/2798481487 learned_addr learned my addr 172.21.15.28:0/2798481487 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f530f02f740 10 client.0 _ll_register_callbacks cb 0x5558d513b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.626+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d5227890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530a75d640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d522ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 -- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 msgr2=0x5558d5227890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d5227890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 -- 172.21.15.28:0/2798481487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 msgr2=0x5558d522ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d522ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 -- 172.21.15.28:0/2798481487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d4e94210 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530975b640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000c00 tx=0x7f52f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb03998f86a232b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f52ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40bc4f0 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f52f40ba7f0 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f40c08a0 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52f40872b0 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530f02f740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558d4f60620 con 0x5558d521e2e0 2024-09-19T09:33:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.627+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f40c08a0 con 0x5558d521e2e0 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f52f40c4cd0 con 0x5558d521e2e0 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 20 client.15684 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d5233300 0x5558d52356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 4 client.15684 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558d518ea20 con 0x5558d5233300 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d5233300 0x5558d52356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d5233300 0x5558d52356f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2eb22d60ae7fd7 server_cookie=46c6e5353cc9af84 in_seq=0 out_seq=0 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.628+0000 7f52ed7fa640 10 client.15684 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f52ed7fa640 1 -- 172.21.15.28:0/2798481487 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558d518ea20 con 0x5558d5233300 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f52ed7fa640 10 client.15684 handle_command_reply: tid=0 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 client.15684 shutdown 2024-09-19T09:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 msgr2=0x5558d5227dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000c00 tx=0x7f52f4009260 comp rx=0 tx=0).stop 2024-09-19T09:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 shutdown_connections 2024-09-19T09:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d522ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d5233300 0x5558d52356f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d5227dd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 --2- 172.21.15.28:0/2798481487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521d9a0 0x5558d5227890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 >> 172.21.15.28:0/2798481487 conn(0x5558d5219010 msgr2=0x5558d521a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.629+0000 7f52ed7fa640 0 client.15684 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.630+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 shutdown_connections 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.630+0000 7f530f02f740 1 -- 172.21.15.28:0/2798481487 wait complete. 2024-09-19T09:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.630+0000 7f530f02f740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T09:33:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 Processor -- start 2024-09-19T09:33:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 -- start start 2024-09-19T09:33:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 0x5558d5219ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d5223370 0x5558d521a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ab240 con 0x5558d521e2e0 2024-09-19T09:33:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53041acd80 con 0x5558d521d9a0 2024-09-19T09:33:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530f02f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ab910 con 0x5558d5223370 2024-09-19T09:33:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530975b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54166/0 (socket says 172.21.15.28:54166) 2024-09-19T09:33:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.658+0000 7f530975b640 1 -- 172.21.15.28:0/2903899189 learned_addr learned my addr 172.21.15.28:0/2903899189 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 0x5558d5219ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530a75d640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d5223370 0x5558d521a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 -- 172.21.15.28:0/2903899189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d5223370 msgr2=0x5558d521a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d5223370 0x5558d521a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 -- 172.21.15.28:0/2903899189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 msgr2=0x5558d5219ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 0x5558d5219ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 -- 172.21.15.28:0/2903899189 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d52142d0 con 0x5558d521e2e0 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f5309f5c640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 0x5558d5219ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f530975b640 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f52f4038020 tx=0x7f52f4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca0c94e8d7f0373 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f52ef7fe640 1 -- 172.21.15.28:0/2903899189 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f409d030 con 0x5558d521e2e0 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f52ef7fe640 1 -- 172.21.15.28:0/2903899189 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f40bc4f0 con 0x5558d521e2e0 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.659+0000 7f52ef7fe640 1 -- 172.21.15.28:0/2903899189 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40bc750 con 0x5558d521e2e0 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- 172.21.15.28:0/2903899189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 msgr2=0x5558d522edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f52f4038020 tx=0x7f52f4032f90 comp rx=0 tx=0).stop 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- 172.21.15.28:0/2903899189 shutdown_connections 2024-09-19T09:33:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d5223370 0x5558d521a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521e2e0 0x5558d522edd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 --2- 172.21.15.28:0/2903899189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d521d9a0 0x5558d5219ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- 172.21.15.28:0/2903899189 >> 172.21.15.28:0/2903899189 conn(0x5558d5219010 msgr2=0x5558d521f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- 172.21.15.28:0/2903899189 shutdown_connections 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- 172.21.15.28:0/2903899189 wait complete. 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 -- start start 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.660+0000 7f530f02f740 1 Processor -- start 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521e2e0 0x5558d522c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d529bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ac6b0 con 0x5558d521d9a0 2024-09-19T09:33:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53041abfe0 con 0x5558d5223370 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53041a9220 con 0x5558d521e2e0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530f02f740 10 client.0 _ll_register_callbacks cb 0x7f52fc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54184/0 (socket says 172.21.15.28:54184) 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 -- 172.21.15.28:0/1750263692 learned_addr learned my addr 172.21.15.28:0/1750263692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530975b640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521e2e0 0x5558d522c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f530a75d640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d529bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 -- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521e2e0 msgr2=0x5558d522c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521e2e0 0x5558d522c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 -- 172.21.15.28:0/1750263692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 msgr2=0x5558d529bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d529bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.661+0000 7f5309f5c640 1 -- 172.21.15.28:0/1750263692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d529c4d0 con 0x5558d521d9a0 2024-09-19T09:33:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f5309f5c640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5304101a90 tx=0x7f53000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c31980c275caf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f52ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300011dd0 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5300004020 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f40bc4f0 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5300004260 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f530f02f740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.662+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558d529d8b0 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f40bc4f0 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5300003ad0 con 0x5558d521d9a0 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 20 client.15690 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d529e1d0 0x5558d52a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 4 client.15690 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558d52230f0 con 0x5558d529e1d0 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.663+0000 7f530975b640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d529e1d0 0x5558d52a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.664+0000 7f530975b640 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d529e1d0 0x5558d52a05f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f25d7d3be9adcd server_cookie=1064fb76e9dee7de in_seq=0 out_seq=0 2024-09-19T09:33:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.664+0000 7f52ed7fa640 10 client.15690 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f52ed7fa640 1 -- 172.21.15.28:0/1750263692 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558d52230f0 con 0x5558d529e1d0 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f52ed7fa640 10 client.15690 handle_command_reply: tid=0 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 client.15690 shutdown 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 msgr2=0x5558d522be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5304101a90 tx=0x7f53000046c0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 shutdown_connections 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558d5223370 0x5558d529bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558d529e1d0 0x5558d52a05f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558d521e2e0 0x5558d522c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f52ed7fa640 0 client.15690 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 --2- 172.21.15.28:0/1750263692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558d521d9a0 0x5558d522be60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 >> 172.21.15.28:0/1750263692 conn(0x5558d5219010 msgr2=0x5558d52179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 shutdown_connections 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 1 -- 172.21.15.28:0/1750263692 wait complete. 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.671+0000 7f530f02f740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T09:33:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:33:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:33:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:33:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:33:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 msgr2=0x7f52f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 0x7f52f00a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f53000058c0 tx=0x7f5300005850 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 msgr2=0x7f5304103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f52f4031440 tx=0x7f52f4032ce0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 shutdown_connections 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f52f00a32d0 0x7f52f00a5790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53041025f0 0x7f5304103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5304101c30 0x7f5304103010 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 --2- 172.21.15.28:0/1989293884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5304100a40 0x7f53041049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 >> 172.21.15.28:0/1989293884 conn(0x7f53040fc1c0 msgr2=0x7f5304069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 shutdown_connections 2024-09-19T09:33:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:28.672+0000 7f530c1c0640 1 -- 172.21.15.28:0/1989293884 wait complete. 2024-09-19T09:33:28.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:28 smithi028 ceph-mon[30972]: pgmap v362: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:28.821 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:33:28.822 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:33:28.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:28 smithi078 ceph-mon[35619]: pgmap v362: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:28 smithi038 ceph-mon[35271]: pgmap v362: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:29.270 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b7b0c68d', total seconds elapsed: 15.6 2024-09-19T09:33:29.351 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.851+0000 7f4418d3e640 1 -- 172.21.15.28:0/823150900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4414102d30 msgr2=0x7f4414103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.851+0000 7f4418d3e640 1 --2- 172.21.15.28:0/823150900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4414102d30 0x7f4414103110 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4404009930 tx=0x7f440402a420 comp rx=0 tx=0).stop 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.852+0000 7f4418d3e640 1 -- 172.21.15.28:0/823150900 shutdown_connections 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.852+0000 7f4418d3e640 1 --2- 172.21.15.28:0/823150900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44141082a0 0x7f441410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.852+0000 7f4418d3e640 1 --2- 172.21.15.28:0/823150900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141036e0 0x7f4414107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.852+0000 7f4418d3e640 1 --2- 172.21.15.28:0/823150900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4414102d30 0x7f4414103110 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.852+0000 7f4418d3e640 1 -- 172.21.15.28:0/823150900 >> 172.21.15.28:0/823150900 conn(0x7f44140fc7d0 msgr2=0x7f44140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.853+0000 7f4418d3e640 1 -- 172.21.15.28:0/823150900 shutdown_connections 2024-09-19T09:33:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.853+0000 7f4418d3e640 1 -- 172.21.15.28:0/823150900 wait complete. 2024-09-19T09:33:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.853+0000 7f4418d3e640 1 Processor -- start 2024-09-19T09:33:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.853+0000 7f4418d3e640 1 -- start start 2024-09-19T09:33:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141082a0 0x7f4414100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 0x7f44141a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44141120a0 con 0x7f44141036e0 2024-09-19T09:33:29.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4414111f20 con 0x7f4414101410 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54200/0 (socket says 172.21.15.28:54200) 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 -- 172.21.15.28:0/2234170440 learned_addr learned my addr 172.21.15.28:0/2234170440 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112220 con 0x7f44141082a0 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 0x7f44141a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4411d9b640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141082a0 0x7f4414100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 -- 172.21.15.28:0/2234170440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141082a0 msgr2=0x7f4414100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141082a0 0x7f4414100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 -- 172.21.15.28:0/2234170440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 msgr2=0x7f44141a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 0x7f44141a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f441259c640 1 -- 172.21.15.28:0/2234170440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141a7090 con 0x7f44141036e0 2024-09-19T09:33:29.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.854+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 0x7f44141a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f441259c640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4404005940 tx=0x7f4404031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f44037fe640 1 -- 172.21.15.28:0/2234170440 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440404d070 con 0x7f44141036e0 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f44037fe640 1 -- 172.21.15.28:0/2234170440 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404004450 con 0x7f44141036e0 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f44037fe640 1 -- 172.21.15.28:0/2234170440 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404035070 con 0x7f44141036e0 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4414198540 con 0x7f44141036e0 2024-09-19T09:33:29.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.855+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4414198a20 con 0x7f44141036e0 2024-09-19T09:33:29.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.858+0000 7f44037fe640 1 -- 172.21.15.28:0/2234170440 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f4404050070 con 0x7f44141036e0 2024-09-19T09:33:29.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.858+0000 7f441b117740 1 Processor -- start 2024-09-19T09:33:29.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 -- start start 2024-09-19T09:33:29.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558fbd524280 con 0x558fbd5bf480 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558fbd4cbb10 con 0x558fbd5c9d40 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f441b117740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbd520240 con 0x558fbd5beb60 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37180/0 (socket says 172.21.15.28:37180) 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.859+0000 7f4412d9d640 1 -- 172.21.15.28:0/2076807403 learned_addr learned my addr 172.21.15.28:0/2076807403 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4411d9b640 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f441259c640 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 -- 172.21.15.28:0/2076807403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 msgr2=0x558fbd5cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 -- 172.21.15.28:0/2076807403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 msgr2=0x558fbd5c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 -- 172.21.15.28:0/2076807403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbd4d81f0 con 0x558fbd5beb60 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f44037fe640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 0x7f43fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f441259c640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 0x7f43fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.860+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f440c011790 tx=0x7f440c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975e1dce6a77b000 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.861+0000 7f4400ff9640 1 -- 172.21.15.28:0/2076807403 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003c30 con 0x558fbd5beb60 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.861+0000 7f4400ff9640 1 -- 172.21.15.28:0/2076807403 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c00fe80 con 0x558fbd5beb60 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.861+0000 7f4400ff9640 1 -- 172.21.15.28:0/2076807403 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0193e0 con 0x558fbd5beb60 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.861+0000 7f44037fe640 1 -- 172.21.15.28:0/2234170440 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f44040f7c20 con 0x7f44141036e0 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 -- 172.21.15.28:0/2076807403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 msgr2=0x558fbd5bef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441259c640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 0x7f43fc0a5810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x558fbd5bbf50 tx=0x7f44040fb770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f440c011790 tx=0x7f440c011c50 comp rx=0 tx=0).stop 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 -- 172.21.15.28:0/2076807403 shutdown_connections 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 --2- 172.21.15.28:0/2076807403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5bef40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 -- 172.21.15.28:0/2076807403 >> 172.21.15.28:0/2076807403 conn(0x558fbd5bb0f0 msgr2=0x558fbd5bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 -- 172.21.15.28:0/2076807403 shutdown_connections 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.862+0000 7f441b117740 1 -- 172.21.15.28:0/2076807403 wait complete. 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.863+0000 7f441b117740 1 -- start start 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.863+0000 7f441b117740 1 Processor -- start 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.863+0000 7f441b117740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.863+0000 7f441b117740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.863+0000 7f441b117740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f441b117740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558fbd5cc910 con 0x558fbd5bf480 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f441b117740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558fbd4def10 con 0x558fbd5c9d40 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f441b117740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbd5cca90 con 0x558fbd5beb60 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f441b117740 10 client.0 _ll_register_callbacks cb 0x558fbd4a7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37186/0 (socket says 172.21.15.28:37186) 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 -- 172.21.15.28:0/2893118505 learned_addr learned my addr 172.21.15.28:0/2893118505 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 -- 172.21.15.28:0/2893118505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 msgr2=0x558fbd5c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 -- 172.21.15.28:0/2893118505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 msgr2=0x558fbd5c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 -- 172.21.15.28:0/2893118505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbd2956a0 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f4412d9d640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f440c0046a0 tx=0x7f440c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c9b58c6437733e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f43f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.864+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004c40 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f440c004de0 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fbd5317f0 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f440c003a30 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f441b117740 10 client.25306 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fbd5d8540 con 0x558fbd5beb60 2024-09-19T09:33:29.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.865+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558fbd5317f0 con 0x558fbd5beb60 2024-09-19T09:33:29.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.866+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f440c018070 con 0x558fbd5beb60 2024-09-19T09:33:29.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.866+0000 7f441b117740 10 client.25306 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:29.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.866+0000 7f441b117740 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:29.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.866+0000 7f441b117740 20 client.25306 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:29.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.867+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd5d9950 0x558fbd5dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.867+0000 7f441b117740 4 client.25306 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.867+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fbd52bd00 con 0x558fbd5d9950 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.867+0000 7f4411d9b640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd5d9950 0x558fbd5dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.868+0000 7f43f9ffb640 10 client.25306 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.868+0000 7f4411d9b640 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd5d9950 0x558fbd5dbd40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbee44c74015ccb9 server_cookie=1d5d35672f45c631 in_seq=0 out_seq=0 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.868+0000 7f43f9ffb640 1 -- 172.21.15.28:0/2893118505 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fbd52bd00 con 0x558fbd5d9950 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.868+0000 7f43f9ffb640 10 client.25306 handle_command_reply: tid=0 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.868+0000 7f441b117740 1 client.25306 shutdown 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 msgr2=0x558fbd5c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f440c0046a0 tx=0x7f440c0045a0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 shutdown_connections 2024-09-19T09:33:29.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd5d9950 0x558fbd5dbd40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f43f9ffb640 0 client.25306 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5bf480 0x558fbd5c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 --2- 172.21.15.28:0/2893118505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5c6be0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 >> 172.21.15.28:0/2893118505 conn(0x558fbd5bb0f0 msgr2=0x558fbd5c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 shutdown_connections 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 1 -- 172.21.15.28:0/2893118505 wait complete. 2024-09-19T09:33:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.869+0000 7f441b117740 20 client.25306 trim_cache size 0 max 16384 2024-09-19T09:33:29.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.903+0000 7f441b117740 1 Processor -- start 2024-09-19T09:33:29.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 -- start start 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 0x558fbd5ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44141ab8a0 con 0x558fbd5beb60 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44141ad3e0 con 0x558fbd5c9d40 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f441b117740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44141abf70 con 0x558fbd5bf480 2024-09-19T09:33:29.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37200/0 (socket says 172.21.15.28:37200) 2024-09-19T09:33:29.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f4411d9b640 1 -- 172.21.15.28:0/3881102755 learned_addr learned my addr 172.21.15.28:0/3881102755 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:29.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.904+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 0x558fbd5ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4412d9d640 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 -- 172.21.15.28:0/3881102755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 msgr2=0x558fbd5e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5e1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 -- 172.21.15.28:0/3881102755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 msgr2=0x558fbd5ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 0x558fbd5ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 -- 172.21.15.28:0/3881102755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbd5dc300 con 0x558fbd5bf480 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4412d9d640 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 0x558fbd5ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.905+0000 7f4411d9b640 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4414102420 tx=0x7f4408012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b3eef4774e016b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.906+0000 7f43fbfff640 1 -- 172.21.15.28:0/3881102755 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408005020 con 0x558fbd5bf480 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.906+0000 7f43fbfff640 1 -- 172.21.15.28:0/3881102755 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44080047a0 con 0x558fbd5bf480 2024-09-19T09:33:29.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.906+0000 7f43fbfff640 1 -- 172.21.15.28:0/3881102755 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408010690 con 0x558fbd5bf480 2024-09-19T09:33:29.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.906+0000 7f441b117740 1 -- 172.21.15.28:0/3881102755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 msgr2=0x558fbd5e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4414102420 tx=0x7f4408012820 comp rx=0 tx=0).stop 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 -- 172.21.15.28:0/3881102755 shutdown_connections 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5c9d40 0x558fbd5e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5bf480 0x558fbd5e1530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 --2- 172.21.15.28:0/3881102755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5beb60 0x558fbd5ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 -- 172.21.15.28:0/3881102755 >> 172.21.15.28:0/3881102755 conn(0x558fbd5bb0f0 msgr2=0x558fbd5b9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 -- 172.21.15.28:0/3881102755 shutdown_connections 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 -- 172.21.15.28:0/3881102755 wait complete. 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 -- start start 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.907+0000 7f441b117740 1 Processor -- start 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f441b117740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f441b117740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 0x558fbd5ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f441b117740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 0x558fbd5ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f4412d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:37208/0 (socket says 172.21.15.28:37208) 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 0x558fbd5ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f4412d9d640 1 -- 172.21.15.28:0/852529844 learned_addr learned my addr 172.21.15.28:0/852529844 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.908+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44141acd10 con 0x558fbd5c9d40 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44141ac640 con 0x558fbd5bf480 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44141a98a0 con 0x558fbd5beb60 2024-09-19T09:33:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4412d9d640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 0x558fbd5ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f441b117740 10 client.0 _ll_register_callbacks cb 0x7f44040fb640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4411d9b640 1 -- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 msgr2=0x558fbd5cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4411d9b640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4411d9b640 1 -- 172.21.15.28:0/852529844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 msgr2=0x558fbd5ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4411d9b640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 0x558fbd5ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4411d9b640 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbd5baf20 con 0x558fbd5bf480 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.909+0000 7f4412d9d640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 0x558fbd5ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f4411d9b640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 0x558fbd5ce2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x558fbd5b6950 tx=0x7f4408009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e64451a385a00fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f43f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408004950 con 0x558fbd5bf480 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44080047a0 con 0x558fbd5bf480 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4408004af0 con 0x558fbd5bf480 2024-09-19T09:33:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4408037050 con 0x558fbd5bf480 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f441b117740 10 client.25370 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fbd6448c0 con 0x558fbd5bf480 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.910+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44080047a0 con 0x558fbd5bf480 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4408026720 con 0x558fbd5bf480 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 10 client.25370 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 20 client.25370 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd644f90 0x558fbd6473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 4 client.25370 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558fbd4a8110 con 0x558fbd644f90 2024-09-19T09:33:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.911+0000 7f4412d9d640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd644f90 0x558fbd6473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.912+0000 7f4412d9d640 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd644f90 0x558fbd6473b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e2ef33c8049170 server_cookie=c76c301901e60f46 in_seq=0 out_seq=0 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.912+0000 7f43f9ffb640 10 client.25370 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.912+0000 7f43f9ffb640 1 -- 172.21.15.28:0/852529844 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fbd4a8110 con 0x558fbd644f90 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.912+0000 7f43f9ffb640 10 client.25370 handle_command_reply: tid=0 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 client.25370 shutdown 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 msgr2=0x558fbd5ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 0x558fbd5ce2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x558fbd5b6950 tx=0x7f4408009eb0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 shutdown_connections 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fbd5bf480 0x558fbd5ce2b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fbd644f90 0x558fbd6473b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fbd5c9d40 0x558fbd5ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 --2- 172.21.15.28:0/852529844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbd5beb60 0x558fbd5cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 >> 172.21.15.28:0/852529844 conn(0x558fbd5bb0f0 msgr2=0x558fbd5c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f43f9ffb640 0 client.25370 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 shutdown_connections 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 1 -- 172.21.15.28:0/852529844 wait complete. 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.913+0000 7f441b117740 20 client.25370 trim_cache size 0 max 16384 2024-09-19T09:33:29.917 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "6d05220a-1496-40ef-a06d-a4990318539a", 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 msgr2=0x7f43fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 0x7f43fc0a5810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x558fbd5bbf50 tx=0x7f44040fb770 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 msgr2=0x7f4414100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4404005940 tx=0x7f4404031860 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 shutdown_connections 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f43fc0a3350 0x7f43fc0a5810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4414101410 0x7f44141a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44141082a0 0x7f4414100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 --2- 172.21.15.28:0/2234170440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44141036e0 0x7f4414100990 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4404005940 tx=0x7f4404031860 comp rx=0 tx=0).stop 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.914+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 >> 172.21.15.28:0/2234170440 conn(0x7f44140fc7d0 msgr2=0x7f4414105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.915+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 shutdown_connections 2024-09-19T09:33:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:29.915+0000 7f4418d3e640 1 -- 172.21.15.28:0/2234170440 wait complete. 2024-09-19T09:33:30.591 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:33:30.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:30 smithi028 ceph-mon[30972]: pgmap v363: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:30.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:30 smithi078 ceph-mon[35619]: pgmap v363: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:30.877 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:30.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:30 smithi038 ceph-mon[35271]: pgmap v363: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.322+0000 7f1ec8658640 1 -- 172.21.15.28:0/3752803417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 msgr2=0x7f1ec01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.322+0000 7f1ec8658640 1 --2- 172.21.15.28:0/3752803417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec01021d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009900 tx=0x7f1ebc02a410 comp rx=0 tx=0).stop 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 -- 172.21.15.28:0/3752803417 shutdown_connections 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 --2- 172.21.15.28:0/3752803417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec0106f20 0x7f1ec010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 --2- 172.21.15.28:0/3752803417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec01027a0 0x7f1ec01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 --2- 172.21.15.28:0/3752803417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec01021d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 -- 172.21.15.28:0/3752803417 >> 172.21.15.28:0/3752803417 conn(0x7f1ec00fd8b0 msgr2=0x7f1ec00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 -- 172.21.15.28:0/3752803417 shutdown_connections 2024-09-19T09:33:31.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.323+0000 7f1ec8658640 1 -- 172.21.15.28:0/3752803417 wait complete. 2024-09-19T09:33:31.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 Processor -- start 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 -- start start 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec01027a0 0x7f1ec0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 0x7f1ec0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0111f80 con 0x7f1ec0101df0 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0111e00 con 0x7f1ec01027a0 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec8658640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0112100 con 0x7f1ec0106f20 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54272/0 (socket says 172.21.15.28:54272) 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec63f4640 1 -- 172.21.15.28:0/4229775648 learned_addr learned my addr 172.21.15.28:0/4229775648 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec01027a0 0x7f1ec0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.324+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 0x7f1ec0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/4229775648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec01027a0 msgr2=0x7f1ec0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec01027a0 0x7f1ec0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 msgr2=0x7f1ec00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/4229775648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec01a6950 con 0x7f1ec0106f20 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 0x7f1ec0074f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800da10 tx=0x7f1eb800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1eb77fe640 1 -- 172.21.15.28:0/4229775648 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb800bd70 con 0x7f1ec0106f20 2024-09-19T09:33:31.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec01a6c40 con 0x7f1ec0106f20 2024-09-19T09:33:31.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1eb77fe640 1 -- 172.21.15.28:0/4229775648 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8003e50 con 0x7f1ec0106f20 2024-09-19T09:33:31.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.325+0000 7f1eb77fe640 1 -- 172.21.15.28:0/4229775648 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80106a0 con 0x7f1ec0106f20 2024-09-19T09:33:31.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.326+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01a7180 con 0x7f1ec0106f20 2024-09-19T09:33:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.328+0000 7f1ecaa31740 1 Processor -- start 2024-09-19T09:33:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1eb77fe640 1 -- 172.21.15.28:0/4229775648 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1eb8003970 con 0x7f1ec0106f20 2024-09-19T09:33:31.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1ecaa31740 1 -- start start 2024-09-19T09:33:31.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 0x565483dd9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483de4680 0x565483de6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.329+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x565483d3f2e0 con 0x565483de4680 2024-09-19T09:33:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x565483ce6ce0 con 0x565483dd9dc0 2024-09-19T09:33:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565483d3b2a0 con 0x565483dd94a0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51392/0 (socket says 172.21.15.28:51392) 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec63f4640 1 -- 172.21.15.28:0/1035040985 learned_addr learned my addr 172.21.15.28:0/1035040985 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483de4680 0x565483de6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 0x565483dd9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 -- 172.21.15.28:0/1035040985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 msgr2=0x565483de4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 -- 172.21.15.28:0/1035040985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483de4680 msgr2=0x565483de6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483de4680 0x565483de6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 -- 172.21.15.28:0/1035040985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565483ccb440 con 0x565483dd94a0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.330+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 0x565483dd9880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009fd0 tx=0x7f1eb0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2143ea0b5e8ef56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.331+0000 7f1eb4ff9640 1 -- 172.21.15.28:0/1035040985 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0011d90 con 0x565483dd94a0 2024-09-19T09:33:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.331+0000 7f1eb77fe640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 0x7f1eac0a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.331+0000 7f1eb4ff9640 1 -- 172.21.15.28:0/1035040985 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb0005500 con 0x565483dd94a0 2024-09-19T09:33:31.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.331+0000 7f1eb4ff9640 1 -- 172.21.15.28:0/1035040985 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000f790 con 0x565483dd94a0 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 0x7f1eac0a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1035040985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 msgr2=0x565483dd9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 0x565483dd9880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009fd0 tx=0x7f1eb0005f90 comp rx=0 tx=0).stop 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1035040985 shutdown_connections 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483de4680 0x565483de6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1035040985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483dd94a0 0x565483dd9880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1035040985 >> 172.21.15.28:0/1035040985 conn(0x565483dd5a30 msgr2=0x565483dd5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1eb77fe640 1 -- 172.21.15.28:0/4229775648 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1eb80cfd10 con 0x7f1ec0106f20 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.332+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 0x7f1eac0a5890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0096c0 tx=0x7f1ebc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.333+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1035040985 shutdown_connections 2024-09-19T09:33:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.333+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1035040985 wait complete. 2024-09-19T09:33:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.333+0000 7f1ecaa31740 1 -- start start 2024-09-19T09:33:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.333+0000 7f1ecaa31740 1 Processor -- start 2024-09-19T09:33:31.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.333+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x565483de7250 con 0x565483dd94a0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x565483cfa050 con 0x565483dd9dc0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565483de73d0 con 0x565483de4680 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ecaa31740 10 client.0 _ll_register_callbacks cb 0x565483d3a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54292/0 (socket says 172.21.15.28:54292) 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 -- 172.21.15.28:0/858856746 learned_addr learned my addr 172.21.15.28:0/858856746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 -- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 msgr2=0x565483de1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 -- 172.21.15.28:0/858856746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 msgr2=0x565483de12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 -- 172.21.15.28:0/858856746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565483ab03c0 con 0x565483dd94a0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.334+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005ab0 tx=0x7f1ebc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583af98e2ec16fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc035040 con 0x565483dd94a0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1ebc004040 con 0x565483dd94a0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb0005500 con 0x565483dd94a0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc004240 con 0x565483dd94a0 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ecaa31740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:31.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565483df2c40 con 0x565483dd94a0 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.335+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb0005500 con 0x565483dd94a0 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1ebc03d450 con 0x565483dd94a0 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 20 client.15717 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483df6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 4 client.15717 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565483d4c850 con 0x565483df4100 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483df6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ea9ffb640 10 client.15717 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.336+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483df6560 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7681212f5550a808 server_cookie=f72a3f7690b3cb6b in_seq=0 out_seq=0 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/858856746 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565483d4c850 con 0x565483df4100 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ea9ffb640 10 client.15717 handle_command_reply: tid=0 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ecaa31740 1 client.15717 shutdown 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 msgr2=0x565483de4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.337+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005ab0 tx=0x7f1ebc009260 comp rx=0 tx=0).stop 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 shutdown_connections 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483de12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483df6560 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ea9ffb640 0 client.15717 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/858856746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483de4010 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 >> 172.21.15.28:0/858856746 conn(0x565483dd5a30 msgr2=0x565483dd4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 shutdown_connections 2024-09-19T09:33:31.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 1 -- 172.21.15.28:0/858856746 wait complete. 2024-09-19T09:33:31.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.338+0000 7f1ecaa31740 20 client.15717 trim_cache size 0 max 16384 2024-09-19T09:33:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 Processor -- start 2024-09-19T09:33:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 -- start start 2024-09-19T09:33:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483ddfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483dfbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ec63f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54308/0 (socket says 172.21.15.28:54308) 2024-09-19T09:33:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01ab350 con 0x565483dd94a0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01ace90 con 0x565483dd9dc0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.373+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01aba20 con 0x565483de4680 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 -- 172.21.15.28:0/1508034551 learned_addr learned my addr 172.21.15.28:0/1508034551 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483ddfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483dfbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 -- 172.21.15.28:0/1508034551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 msgr2=0x565483dfbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483dfbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 -- 172.21.15.28:0/1508034551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 msgr2=0x565483ddfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483ddfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 -- 172.21.15.28:0/1508034551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565483de0270 con 0x565483dd94a0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.374+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc03d870 tx=0x7f1ebc032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214ca60cd9210a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.375+0000 7f1eabfff640 1 -- 172.21.15.28:0/1508034551 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc03d450 con 0x565483dd94a0 2024-09-19T09:33:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.375+0000 7f1eabfff640 1 -- 172.21.15.28:0/1508034551 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc03d5f0 con 0x565483dd94a0 2024-09-19T09:33:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.375+0000 7f1eabfff640 1 -- 172.21.15.28:0/1508034551 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0046c0 con 0x565483dd94a0 2024-09-19T09:33:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.375+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 msgr2=0x565483df04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.375+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc03d870 tx=0x7f1ebc032e80 comp rx=0 tx=0).stop 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 shutdown_connections 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd9dc0 0x565483ddfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483dfbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/1508034551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd94a0 0x565483df04b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 >> 172.21.15.28:0/1508034551 conn(0x565483dd5a30 msgr2=0x565483ddb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 shutdown_connections 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 -- 172.21.15.28:0/1508034551 wait complete. 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.376+0000 7f1ecaa31740 1 -- start start 2024-09-19T09:33:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 Processor -- start 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd94a0 0x565483de8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01ac7c0 con 0x565483dd9dc0 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01ac0f0 con 0x565483dd94a0 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01a9330 con 0x565483de4680 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ecaa31740 10 client.0 _ll_register_callbacks cb 0x7f1ebc031680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ec5bf3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ec5bf3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54310/0 (socket says 172.21.15.28:54310) 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ec5bf3640 1 -- 172.21.15.28:0/2600866329 learned_addr learned my addr 172.21.15.28:0/2600866329 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.377+0000 7f1ec6bf5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/2600866329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd94a0 msgr2=0x565483de8310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd94a0 0x565483de8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/2600866329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 msgr2=0x565483de8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 -- 172.21.15.28:0/2600866329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565483e5de00 con 0x565483de4680 2024-09-19T09:33:31.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec5bf3640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:33:31.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ec6bf5640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de8d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80cd690 tx=0x7f1eb809c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944790d37bce7b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80b4020 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1eb8098810 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ebc03d5f0 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1eb8003c60 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ecaa31740 10 client.25327 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.378+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565483e5f0f0 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc03d5f0 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1eb8024060 con 0x565483de4680 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 10 client.25327 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 20 client.25327 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483e61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 4 client.25327 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565483d46d60 con 0x565483df4100 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483e61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.379+0000 7f1ea9ffb640 10 client.25327 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ec63f4640 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483e61800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96599883cef407ec server_cookie=6be90f10f415c4d9 in_seq=0 out_seq=0 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ea9ffb640 1 -- 172.21.15.28:0/2600866329 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565483d46d60 con 0x565483df4100 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ea9ffb640 10 client.25327 handle_command_reply: tid=0 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 client.25327 shutdown 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 msgr2=0x565483de8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de8d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80cd690 tx=0x7f1eb809c260 comp rx=0 tx=0).stop 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 shutdown_connections 2024-09-19T09:33:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x565483df4100 0x565483e61800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565483de4680 0x565483de8d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ea9ffb640 0 client.25327 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565483dd9dc0 0x565483de8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 --2- 172.21.15.28:0/2600866329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565483dd94a0 0x565483de8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 >> 172.21.15.28:0/2600866329 conn(0x565483dd5a30 msgr2=0x565483ddb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2024-09-19T09:33:31.385 INFO:teuthology.orchestra.run.smithi028.stdout: "6d05220a-1496-40ef-a06d-a4990318539a": { 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "6d05220a-1496-40ef-a06d-a4990318539a", 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive,force" 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 shutdown_connections 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 1 -- 172.21.15.28:0/2600866329 wait complete. 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.380+0000 7f1ecaa31740 20 client.25327 trim_cache size 0 max 16384 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 msgr2=0x7f1eac0a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 0x7f1eac0a5890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0096c0 tx=0x7f1ebc005a20 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 msgr2=0x7f1ec0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 0x7f1ec0074f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800da10 tx=0x7f1eb800ded0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 shutdown_connections 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1eac0a33d0 0x7f1eac0a5890 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ec01027a0 0x7f1ec0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ec0106f20 0x7f1ec0074f70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 --2- 172.21.15.28:0/4229775648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ec0101df0 0x7f1ec00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 >> 172.21.15.28:0/4229775648 conn(0x7f1ec00fd8b0 msgr2=0x7f1ec00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 shutdown_connections 2024-09-19T09:33:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:31.381+0000 7f1ec8658640 1 -- 172.21.15.28:0/4229775648 wait complete. 2024-09-19T09:33:31.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:31 smithi028 ceph-mon[30972]: scrub queued for path: / 2024-09-19T09:33:31.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:31 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:31.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:31 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:33:31.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:31 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:31 smithi078 ceph-mon[35619]: scrub queued for path: / 2024-09-19T09:33:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:31 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:31 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:33:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:31 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:31.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:31 smithi038 ceph-mon[35271]: scrub queued for path: / 2024-09-19T09:33:31.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:31 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:31.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:31 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:33:31.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:31 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:33:32.090 INFO:tasks.cephfs.filesystem:scrub status for tag:6d05220a-1496-40ef-a06d-a4990318539a - {'path': '/', 'tag': '6d05220a-1496-40ef-a06d-a4990318539a', 'options': 'recursive,force'} 2024-09-19T09:33:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:32 smithi078 ceph-mon[35619]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:32 smithi038 ceph-mon[35271]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:32 smithi028 ceph-mon[30972]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:33:34.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:34 smithi078 ceph-mon[35619]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:33:34.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:34 smithi038 ceph-mon[35271]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:33:35.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:34 smithi028 ceph-mon[30972]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:33:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:35 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:33:35.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:35 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:33:36.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:35 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:33:36.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:36 smithi078 ceph-mon[35619]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:33:36.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:36 smithi038 ceph-mon[35271]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:33:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:36 smithi028 ceph-mon[30972]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-19T09:33:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:38 smithi078 ceph-mon[35619]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:33:38.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:38 smithi038 ceph-mon[35271]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:33:39.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:38 smithi028 ceph-mon[30972]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:33:40.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:40 smithi078 ceph-mon[35619]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T09:33:40.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:40 smithi038 ceph-mon[35271]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T09:33:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:40 smithi028 ceph-mon[30972]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T09:33:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/888769830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 msgr2=0x7fe9e8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/888769830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e8108e80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009ca0 tx=0x7fe9e002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/888769830 shutdown_connections 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/888769830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/888769830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e8108e80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/888769830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 0x7fe9e8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/888769830 >> 172.21.15.28:0/888769830 conn(0x7fe9e80fc1c0 msgr2=0x7fe9e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/888769830 shutdown_connections 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.779+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/888769830 wait complete. 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 Processor -- start 2024-09-19T09:33:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 -- start start 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 0x7fe9e806a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.780+0000 7fe9ee2ba640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810c6a0 con 0x7fe9e81025f0 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34324/0 (socket says 172.21.15.28:34324) 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9ec857640 1 -- 172.21.15.28:0/2921752639 learned_addr learned my addr 172.21.15.28:0/2921752639 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 0x7fe9e806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810c520 con 0x7fe9e8101c30 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e810c820 con 0x7fe9e8100a40 2024-09-19T09:33:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 -- 172.21.15.28:0/2921752639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 msgr2=0x7fe9e806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 0x7fe9e806a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 -- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 msgr2=0x7fe9e819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 -- 172.21.15.28:0/2921752639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e819fa70 con 0x7fe9e8101c30 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.781+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e819cb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00e9f0 tx=0x7fe9dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.782+0000 7fe9e57fa640 1 -- 172.21.15.28:0/2921752639 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00ccc0 con 0x7fe9e8101c30 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.782+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e819fd00 con 0x7fe9e8101c30 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.782+0000 7fe9e57fa640 1 -- 172.21.15.28:0/2921752639 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc010910 con 0x7fe9e8101c30 2024-09-19T09:33:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.782+0000 7fe9e57fa640 1 -- 172.21.15.28:0/2921752639 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc018960 con 0x7fe9e8101c30 2024-09-19T09:33:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.782+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81a0240 con 0x7fe9e8101c30 2024-09-19T09:33:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.784+0000 7fe9e57fa640 1 -- 172.21.15.28:0/2921752639 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fe9dc010ac0 con 0x7fe9e8101c30 2024-09-19T09:33:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.785+0000 7fe9e57fa640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 0x7fe9d00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.785+0000 7fe9f105b740 1 Processor -- start 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.785+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 0x7fe9d00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.785+0000 7fe9e57fa640 1 -- 172.21.15.28:0/2921752639 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe9dc0cfd60 con 0x7fe9e8101c30 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.785+0000 7fe9f105b740 1 -- start start 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f34f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f34f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b2f34532f0 con 0x55b2f34f9780 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b2f33fb270 con 0x55b2f34f3db0 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9f105b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b2f344f2b0 con 0x55b2f34f46f0 2024-09-19T09:33:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34328/0 (socket says 172.21.15.28:34328) 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 -- 172.21.15.28:0/4279311657 learned_addr learned my addr 172.21.15.28:0/4279311657 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f34f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 0x7fe9d00a5810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005fd0 tx=0x7fe9d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f34f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 -- 172.21.15.28:0/4279311657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 msgr2=0x55b2f34f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f34f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 -- 172.21.15.28:0/4279311657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 msgr2=0x55b2f34f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f34f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.786+0000 7fe9ec857640 1 -- 172.21.15.28:0/4279311657 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f33fa340 con 0x55b2f34f9780 2024-09-19T09:33:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.787+0000 7fe9ec857640 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00049d0 tx=0x7fe9e0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a70813356a36238 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.787+0000 7fe9ce7fc640 1 -- 172.21.15.28:0/4279311657 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0049070 con 0x55b2f34f9780 2024-09-19T09:33:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.787+0000 7fe9ce7fc640 1 -- 172.21.15.28:0/4279311657 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e0036040 con 0x55b2f34f9780 2024-09-19T09:33:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.787+0000 7fe9ce7fc640 1 -- 172.21.15.28:0/4279311657 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e002fc90 con 0x55b2f34f9780 2024-09-19T09:33:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- 172.21.15.28:0/4279311657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 msgr2=0x55b2f34fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00049d0 tx=0x7fe9e0005ef0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- 172.21.15.28:0/4279311657 shutdown_connections 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f34fbc40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f34f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4279311657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f34f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- 172.21.15.28:0/4279311657 >> 172.21.15.28:0/4279311657 conn(0x55b2f34eb010 msgr2=0x55b2f34eb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- 172.21.15.28:0/4279311657 shutdown_connections 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- 172.21.15.28:0/4279311657 wait complete. 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.788+0000 7fe9f105b740 1 -- start start 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 Processor -- start 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f46f0 0x55b2f34fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3505320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b2f34fc570 con 0x55b2f34f9780 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b2f33dc290 con 0x55b2f34f46f0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b2f34fc6f0 con 0x55b2f34f3db0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3505320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f46f0 0x55b2f34fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36558/0 (socket says 172.21.15.28:36558) 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9e7fff640 1 -- 172.21.15.28:0/677754307 learned_addr learned my addr 172.21.15.28:0/677754307 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.789+0000 7fe9f105b740 10 client.0 _ll_register_callbacks cb 0x55b2f340d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 -- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 msgr2=0x55b2f34fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 -- 172.21.15.28:0/677754307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f46f0 msgr2=0x55b2f34fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f46f0 0x55b2f34fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 -- 172.21.15.28:0/677754307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f3166210 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e0036040 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9ec857640 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3505320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa8a87582f8433e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9c77fe640 10 client.15735 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9c77fe640 1 -- 172.21.15.28:0/677754307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0049070 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9c77fe640 1 -- 172.21.15.28:0/677754307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe9e0003970 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9c77fe640 1 -- 172.21.15.28:0/677754307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9e0003b70 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9f105b740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.790+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55b2f3232620 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9c77fe640 1 -- 172.21.15.28:0/677754307 <== mon.0 v2:172.21.15.28:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe9e002f9d0 con 0x55b2f34f9780 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 20 client.15735 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3509520 0x55b2f350b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 4 client.15735 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2f3460a20 con 0x55b2f3509520 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.791+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3509520 0x55b2f350b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.792+0000 7fe9c77fe640 10 client.15735 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.792+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3509520 0x55b2f350b910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479b1c8f92750007 server_cookie=372905c0efce28ab in_seq=0 out_seq=0 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.792+0000 7fe9c77fe640 1 -- 172.21.15.28:0/677754307 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2f3460a20 con 0x55b2f3509520 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.792+0000 7fe9c77fe640 10 client.15735 handle_command_reply: tid=0 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.792+0000 7fe9f105b740 1 client.15735 shutdown 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 msgr2=0x55b2f3505320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3505320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e002f610 comp rx=0 tx=0).stop 2024-09-19T09:33:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 shutdown_connections 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f46f0 0x55b2f34fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3509520 0x55b2f350b910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9c77fe640 0 client.15735 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3505320 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 --2- 172.21.15.28:0/677754307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 >> 172.21.15.28:0/677754307 conn(0x55b2f34eb010 msgr2=0x55b2f34ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 shutdown_connections 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 1 -- 172.21.15.28:0/677754307 wait complete. 2024-09-19T09:33:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.793+0000 7fe9f105b740 20 client.15735 trim_cache size 0 max 16384 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.807+0000 7fe9f105b740 1 Processor -- start 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 -- start start 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f9780 0x55b2f34e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a6cc0 con 0x55b2f34f46f0 2024-09-19T09:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a8800 con 0x55b2f34f9780 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34350/0 (socket says 172.21.15.28:34350) 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 -- 172.21.15.28:0/4272814054 learned_addr learned my addr 172.21.15.28:0/4272814054 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a7390 con 0x55b2f34f3db0 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 -- 172.21.15.28:0/4272814054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 msgr2=0x55b2f34e99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9ec857640 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f9780 0x55b2f34e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34e99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 -- 172.21.15.28:0/4272814054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f9780 msgr2=0x55b2f34e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f9780 0x55b2f34e9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.808+0000 7fe9e77fe640 1 -- 172.21.15.28:0/4272814054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f350bf70 con 0x55b2f34f46f0 2024-09-19T09:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.809+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0cd710 tx=0x7fe9dc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ca4cfe1de1878e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.809+0000 7fe9cdffb640 1 -- 172.21.15.28:0/4272814054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b5070 con 0x55b2f34f46f0 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.809+0000 7fe9cdffb640 1 -- 172.21.15.28:0/4272814054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc0cd930 con 0x55b2f34f46f0 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.809+0000 7fe9cdffb640 1 -- 172.21.15.28:0/4272814054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0a49d0 con 0x55b2f34f46f0 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 msgr2=0x55b2f3505240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0cd710 tx=0x7fe9dc01a040 comp rx=0 tx=0).stop 2024-09-19T09:33:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 shutdown_connections 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f9780 0x55b2f34e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f46f0 0x55b2f3505240 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 --2- 172.21.15.28:0/4272814054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f3db0 0x55b2f34e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 >> 172.21.15.28:0/4272814054 conn(0x55b2f34eb010 msgr2=0x55b2f34fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 shutdown_connections 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- 172.21.15.28:0/4272814054 wait complete. 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 -- start start 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.810+0000 7fe9f105b740 1 Processor -- start 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f35025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f3502b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a8130 con 0x55b2f34f9780 2024-09-19T09:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a7a60 con 0x55b2f34f3db0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81a4ca0 con 0x55b2f34f46f0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9f105b740 10 client.0 _ll_register_callbacks cb 0x7fe9e0047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34352/0 (socket says 172.21.15.28:34352) 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 -- 172.21.15.28:0/1327265742 learned_addr learned my addr 172.21.15.28:0/1327265742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f35025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9e77fe640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f3502b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 -- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 msgr2=0x55b2f3502b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f3502b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 -- 172.21.15.28:0/1327265742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 msgr2=0x55b2f35025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f35025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 -- 172.21.15.28:0/1327265742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f3504130 con 0x55b2f34f9780 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9ec857640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0005e30 tx=0x7fe9e0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfb8b308ef1d4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.811+0000 7fe9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e002fed0 con 0x55b2f34f9780 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fe9e00307d0 con 0x55b2f34f9780 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9dc0cd930 con 0x55b2f34f9780 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9e00309d0 con 0x55b2f34f9780 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9f105b740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2f3505440 con 0x55b2f34f9780 2024-09-19T09:33:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.812+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc0cd930 con 0x55b2f34f9780 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe9e004fc20 con 0x55b2f34f9780 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 20 client.15741 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3574810 0x55b2f3576c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 4 client.15741 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.813+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2f345ad70 con 0x55b2f3574810 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.814+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3574810 0x55b2f3576c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.814+0000 7fe9c77fe640 10 client.15741 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.814+0000 7fe9e7fff640 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3574810 0x55b2f3576c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c98fc942d796fd3 server_cookie=d90024dfccecb7d4 in_seq=0 out_seq=0 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9c77fe640 1 -- 172.21.15.28:0/1327265742 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2f345ad70 con 0x55b2f3574810 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9c77fe640 10 client.15741 handle_command_reply: tid=0 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 client.15741 shutdown 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 msgr2=0x55b2f3503b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0005e30 tx=0x7fe9e0005d80 comp rx=0 tx=0).stop 2024-09-19T09:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 shutdown_connections 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b2f3574810 0x55b2f3576c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9c77fe640 0 client.15741 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b2f34f9780 0x55b2f3503b20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f34f46f0 0x55b2f3502b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 --2- 172.21.15.28:0/1327265742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b2f34f3db0 0x55b2f35025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 >> 172.21.15.28:0/1327265742 conn(0x55b2f34eb010 msgr2=0x55b2f34ebe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 shutdown_connections 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 1 -- 172.21.15.28:0/1327265742 wait complete. 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.818+0000 7fe9f105b740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T09:33:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 msgr2=0x7fe9d00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 0x7fe9d00a5810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005fd0 tx=0x7fe9d800b410 comp rx=0 tx=0).stop 2024-09-19T09:33:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 msgr2=0x7fe9e819cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e819cb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00e9f0 tx=0x7fe9dc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 shutdown_connections 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe9d00a3350 0x7fe9d00a5810 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e8101c30 0x7fe9e819cb40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9e81025f0 0x7fe9e819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 --2- 172.21.15.28:0/2921752639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e8100a40 0x7fe9e806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 >> 172.21.15.28:0/2921752639 conn(0x7fe9e80fc1c0 msgr2=0x7fe9e8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 shutdown_connections 2024-09-19T09:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:41.819+0000 7fe9ee2ba640 1 -- 172.21.15.28:0/2921752639 wait complete. 2024-09-19T09:33:42.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:42 smithi078 ceph-mon[35619]: pgmap v369: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:33:42.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:42 smithi038 ceph-mon[35271]: pgmap v369: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:33:43.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:42 smithi028 ceph-mon[30972]: pgmap v369: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T09:33:44.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:44 smithi078 ceph-mon[35619]: pgmap v370: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T09:33:44.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:44 smithi038 ceph-mon[35271]: pgmap v370: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T09:33:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:44 smithi028 ceph-mon[30972]: pgmap v370: 97 pgs: 97 active+clean; 674 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-19T09:33:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:46 smithi078 ceph-mon[35619]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:33:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:33:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:46.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:46 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:46.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:46 smithi038 ceph-mon[35271]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:33:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:33:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:46 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:46 smithi028 ceph-mon[30972]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:33:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:33:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:46 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:33:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:33:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:47.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:33:47.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:33:47.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:33:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:33:48.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:33:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:48 smithi028 ceph-mon[30972]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:33:49.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:48 smithi028 ceph-mon[30972]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:33:49.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:48 smithi078 ceph-mon[35619]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:33:49.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:48 smithi078 ceph-mon[35619]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:33:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:48 smithi038 ceph-mon[35271]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:33:49.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:48 smithi038 ceph-mon[35271]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:33:49.737 INFO:tasks.workunit.client.0.smithi028.stderr:Updating files: 95% (12349/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T09:33:50.070 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:33:50.071 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:33:50.071 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:33:50.330 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:50.658 INFO:tasks.workunit.client.0.smithi028.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T09:33:50.658 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.658 INFO:tasks.workunit.client.0.smithi028.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr: git switch -c 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr:Or undo this operation with: 2024-09-19T09:33:50.659 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.660 INFO:tasks.workunit.client.0.smithi028.stderr: git switch - 2024-09-19T09:33:50.660 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.660 INFO:tasks.workunit.client.0.smithi028.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T09:33:50.660 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:50.660 INFO:tasks.workunit.client.0.smithi028.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T09:33:50.665 DEBUG:teuthology.orchestra.run.smithi028:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T09:33:50.705 INFO:tasks.workunit.client.0.smithi028.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T09:33:50.708 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:33:50.708 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T09:33:50.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3521158124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34102ef0 msgr2=0x7f5a3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:50.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3521158124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34102ef0 0x7f5a3410da40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009ca0 tx=0x7f5a2c02a950 comp rx=0 tx=0).stop 2024-09-19T09:33:50.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3521158124 shutdown_connections 2024-09-19T09:33:50.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3521158124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 0x7f5a341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3521158124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34102ef0 0x7f5a3410da40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3521158124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34100e20 0x7f5a34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.991+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3521158124 >> 172.21.15.28:0/3521158124 conn(0x7f5a340fc7d0 msgr2=0x7f5a340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.992+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3521158124 shutdown_connections 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.992+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3521158124 wait complete. 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.992+0000 7f5a3b14e640 1 Processor -- start 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.992+0000 7f5a3b14e640 1 -- start start 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 0x7f5a34199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:51.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34102ef0 0x7f5a341a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34110fd0 con 0x7f5a34100e20 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34110e50 con 0x7f5a34102570 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a3b14e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111150 con 0x7f5a34102ef0 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48740/0 (socket says 172.21.15.28:48740) 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 -- 172.21.15.28:0/3630580072 learned_addr learned my addr 172.21.15.28:0/3630580072 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a33fff640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 0x7f5a34199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a396eb640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34102ef0 0x7f5a341a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 -- 172.21.15.28:0/3630580072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34102ef0 msgr2=0x7f5a341a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:51.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34102ef0 0x7f5a341a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 -- 172.21.15.28:0/3630580072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 msgr2=0x7f5a34199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 0x7f5a34199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 -- 172.21.15.28:0/3630580072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a341a48c0 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a38eea640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d950 tx=0x7f5a2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.993+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28017030 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.994+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a28004020 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.994+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341a4a90 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.994+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a280047a0 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.994+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a341a4fd0 con 0x7f5a34100e20 2024-09-19T09:33:51.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.995+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34101200 con 0x7f5a34100e20 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.996+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5a280041d0 con 0x7f5a34100e20 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.997+0000 7f5a31ffb640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 0x7f5a240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.997+0000 7f5a33fff640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 0x7f5a240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.997+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5a280cf210 con 0x7f5a34100e20 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:50.997+0000 7f5a33fff640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 0x7f5a240a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a3406ab00 tx=0x7f5a20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:51.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.001+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5a2809bf60 con 0x7f5a34100e20 2024-09-19T09:33:51.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.216+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a341a5260 con 0x7f5a34100e20 2024-09-19T09:33:51.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.216+0000 7f5a31ffb640 1 -- 172.21.15.28:0/3630580072 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f5a2809b860 con 0x7f5a34100e20 2024-09-19T09:33:51.218 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:33:51.218 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:33:51.218 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 msgr2=0x7f5a240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 0x7f5a240a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a3406ab00 tx=0x7f5a20005f90 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 msgr2=0x7f5a34199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d950 tx=0x7f5a2800de10 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 shutdown_connections 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5a240a3240 0x7f5a240a5700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a34102570 0x7f5a34199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a34102ef0 0x7f5a341a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 --2- 172.21.15.28:0/3630580072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5a34100e20 0x7f5a34199300 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 >> 172.21.15.28:0/3630580072 conn(0x7f5a340fc7d0 msgr2=0x7f5a34106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:51.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 shutdown_connections 2024-09-19T09:33:51.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:51.219+0000 7f5a3b14e640 1 -- 172.21.15.28:0/3630580072 wait complete. 2024-09-19T09:33:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:50 smithi028 ceph-mon[30972]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:33:51.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:50 smithi078 ceph-mon[35619]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:33:51.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:50 smithi038 ceph-mon[35271]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:33:52.176 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:33:52.176 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:33:52.263 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:51 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3630580072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:52.283 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T09:33:52.354 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T09:33:52.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:51 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3630580072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:52.396 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:33:52.397 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:33:52.397 INFO:tasks.workunit.client.0.smithi028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T09:33:52.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:51 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3630580072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:33:52.440 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:33:52.444 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:33:52.444 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T09:33:52.461 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-19T09:33:52.462 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-19T09:33:52.462 DEBUG:teuthology.orchestra.run.smithi028:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-19T09:33:52.483 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:33:52.509 INFO:tasks.workunit.client.0.smithi028.stdout:getting blogbench 2024-09-19T09:33:52.509 INFO:tasks.workunit.client.0.smithi028.stderr:+ echo 'getting blogbench' 2024-09-19T09:33:52.509 INFO:tasks.workunit.client.0.smithi028.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-19T09:33:52.575 INFO:tasks.workunit.client.0.smithi028.stderr:--2024-09-19 09:33:52-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-19T09:33:52.585 INFO:tasks.workunit.client.0.smithi028.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-19T09:33:52.649 INFO:tasks.workunit.client.0.smithi028.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-19T09:33:52.714 INFO:tasks.workunit.client.0.smithi028.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-19T09:33:52.714 INFO:tasks.workunit.client.0.smithi028.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-19T09:33:52.715 INFO:tasks.workunit.client.0.smithi028.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-19T09:33:52.718 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:52.847 INFO:tasks.workunit.client.0.smithi028.stderr: 0K .......... .......... .......... .......... .......... 49% 385K 0s 2024-09-19T09:33:52.915 INFO:tasks.workunit.client.0.smithi028.stderr: 50K .......... .......... .......... .......... .......... 99% 725K 0s 2024-09-19T09:33:52.915 INFO:tasks.workunit.client.0.smithi028.stderr: 100K 100% 28.6M=0.2s 2024-09-19T09:33:52.915 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:52.916 INFO:tasks.workunit.client.0.smithi028.stderr:2024-09-19 09:33:52 (507 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-19T09:33:52.916 INFO:tasks.workunit.client.0.smithi028.stderr: 2024-09-19T09:33:52.916 INFO:tasks.workunit.client.0.smithi028.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-19T09:33:52.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 -- 172.21.15.28:0/1540986224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 msgr2=0x7f11b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/1540986224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b41067e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f119c009900 tx=0x7f119c02a410 comp rx=0 tx=0).stop 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 -- 172.21.15.28:0/1540986224 shutdown_connections 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/1540986224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 0x7f11b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/1540986224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b41067e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/1540986224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 -- 172.21.15.28:0/1540986224 >> 172.21.15.28:0/1540986224 conn(0x7f11b40fd8c0 msgr2=0x7f11b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 -- 172.21.15.28:0/1540986224 shutdown_connections 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.911+0000 7f11b8ee8640 1 -- 172.21.15.28:0/1540986224 wait complete. 2024-09-19T09:33:52.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 Processor -- start 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 -- start start 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 0x7f11b419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4111f70 con 0x7f11b4102790 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4111df0 con 0x7f11b4106f10 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b8ee8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41120f0 con 0x7f11b4101de0 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48758/0 (socket says 172.21.15.28:48758) 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.912+0000 7f11b1d9b640 1 -- 172.21.15.28:0/4186622055 learned_addr learned my addr 172.21.15.28:0/4186622055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b2d9d640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 0x7f11b419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b259c640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 -- 172.21.15.28:0/4186622055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 msgr2=0x7f11b419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 -- 172.21.15.28:0/4186622055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 msgr2=0x7f11b419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 0x7f11b419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 -- 172.21.15.28:0/4186622055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b419f600 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b259c640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11ab7fe640 1 -- 172.21.15.28:0/4186622055 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c03f030 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11ab7fe640 1 -- 172.21.15.28:0/4186622055 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c003ce0 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b419f890 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11ab7fe640 1 -- 172.21.15.28:0/4186622055 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035070 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.913+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b41a71d0 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11ab7fe640 1 -- 172.21.15.28:0/4186622055 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f119c042030 con 0x7f11b4102790 2024-09-19T09:33:52.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 Processor -- start 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 -- start start 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d844d850 0x5581d844dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d8458a30 0x5581d845ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5581d83b2160 con 0x5581d844e170 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5581d8359720 con 0x5581d8458a30 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.916+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581d83ae120 con 0x5581d844d850 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48764/0 (socket says 172.21.15.28:48764) 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 -- 172.21.15.28:0/3006021130 learned_addr learned my addr 172.21.15.28:0/3006021130 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b2d9d640 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d844d850 0x5581d844dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d8458a30 0x5581d845ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11ab7fe640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 0x7f11a00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 -- 172.21.15.28:0/3006021130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d844d850 msgr2=0x5581d844dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d844d850 0x5581d844dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 -- 172.21.15.28:0/3006021130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d8458a30 msgr2=0x5581d845ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d8458a30 0x5581d845ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 -- 172.21.15.28:0/3006021130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d8123ba0 con 0x5581d844e170 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b2d9d640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 0x7f11a00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11ab7fe640 1 -- 172.21.15.28:0/4186622055 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f119c0efae0 con 0x7f11b4102790 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.917+0000 7f11b259c640 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009c50 tx=0x7f11a400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f5e3eb0ee444e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11a8ff9640 1 -- 172.21.15.28:0/3006021130 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4016070 con 0x5581d844e170 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11a8ff9640 1 -- 172.21.15.28:0/3006021130 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4005500 con 0x5581d844e170 2024-09-19T09:33:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11a8ff9640 1 -- 172.21.15.28:0/3006021130 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a40116d0 con 0x5581d844e170 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3006021130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 msgr2=0x5581d84584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009c50 tx=0x7f11a400e6c0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11b2d9d640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 0x7f11a00a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005fd0 tx=0x7f11ac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3006021130 shutdown_connections 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d8458a30 0x5581d845ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d84584f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3006021130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d844d850 0x5581d844dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.918+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3006021130 >> 172.21.15.28:0/3006021130 conn(0x5581d8449de0 msgr2=0x5581d844a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3006021130 shutdown_connections 2024-09-19T09:33:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3006021130 wait complete. 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- start start 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 Processor -- start 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d8455910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8456390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5581d845b600 con 0x5581d844e170 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5581d833a900 con 0x5581d844d850 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.919+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581d845b780 con 0x5581d8458a30 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48776/0 (socket says 172.21.15.28:48776) 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3943002827 learned_addr learned my addr 172.21.15.28:0/3943002827 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11bb2c1740 10 client.0 _ll_register_callbacks cb 0x5581d8338780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b259c640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d8455910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 msgr2=0x5581d8456390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8456390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3943002827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 msgr2=0x5581d8455910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d8455910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3943002827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d837ac50 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f119c0bbef0 tx=0x7f119c0b8490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9de3a7fb268c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f1199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c049070 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f119c0b8860 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.920+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a4005500 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f119c0bbb50 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 10 client.15759 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581d846fac0 con 0x5581d844e170 2024-09-19T09:33:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4005500 con 0x5581d844e170 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f119c0bc450 con 0x5581d844e170 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 10 client.15759 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 20 client.15759 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d8470ed0 0x5581d8473370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 4 client.15759 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d83bf890 con 0x5581d8470ed0 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.921+0000 7f11b259c640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d8470ed0 0x5581d8473370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.922+0000 7f11b259c640 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d8470ed0 0x5581d8473370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f4de44fc884557 server_cookie=ef419711b947671c in_seq=0 out_seq=0 2024-09-19T09:33:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.922+0000 7f1199ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.922+0000 7f1199ffb640 1 -- 172.21.15.28:0/3943002827 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581d83bf890 con 0x5581d8470ed0 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.922+0000 7f1199ffb640 10 client.15759 handle_command_reply: tid=0 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 client.15759 shutdown 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 msgr2=0x5581d8455e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f119c0bbef0 tx=0x7f119c0b8490 comp rx=0 tx=0).stop 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 shutdown_connections 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d8470ed0 0x5581d8473370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8456390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8455e50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3943002827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d8455910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 >> 172.21.15.28:0/3943002827 conn(0x5581d8449de0 msgr2=0x5581d844fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f1199ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 shutdown_connections 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3943002827 wait complete. 2024-09-19T09:33:52.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.923+0000 7f11bb2c1740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 Processor -- start 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 -- start start 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 0x5581d8479180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8452b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41ab2d0 con 0x5581d844d850 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41ace10 con 0x5581d844e170 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41ab9a0 con 0x5581d8458a30 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11b259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48790/0 (socket says 172.21.15.28:48790) 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.940+0000 7f11b259c640 1 -- 172.21.15.28:0/1397714840 learned_addr learned my addr 172.21.15.28:0/1397714840 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 -- 172.21.15.28:0/1397714840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 msgr2=0x5581d8452b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8452b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 0x5581d8479180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 -- 172.21.15.28:0/1397714840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 msgr2=0x5581d8479180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 0x5581d8479180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 -- 172.21.15.28:0/1397714840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d84548c0 con 0x5581d844d850 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 0x5581d8479180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f11b259c640 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f11a40095a0 tx=0x7f11a4009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540dcdbb5c45c784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f119bfff640 1 -- 172.21.15.28:0/1397714840 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4016040 con 0x5581d844d850 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f119bfff640 1 -- 172.21.15.28:0/1397714840 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4009d40 con 0x5581d844d850 2024-09-19T09:33:52.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.941+0000 7f119bfff640 1 -- 172.21.15.28:0/1397714840 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4011470 con 0x5581d844d850 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- 172.21.15.28:0/1397714840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 msgr2=0x5581d8478c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f11a40095a0 tx=0x7f11a4009380 comp rx=0 tx=0).stop 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- 172.21.15.28:0/1397714840 shutdown_connections 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844e170 0x5581d8479180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8452b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/1397714840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844d850 0x5581d8478c40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- 172.21.15.28:0/1397714840 >> 172.21.15.28:0/1397714840 conn(0x5581d8449de0 msgr2=0x5581d8457990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:52.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- 172.21.15.28:0/1397714840 shutdown_connections 2024-09-19T09:33:52.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- 172.21.15.28:0/1397714840 wait complete. 2024-09-19T09:33:52.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 -- start start 2024-09-19T09:33:52.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.942+0000 7f11bb2c1740 1 Processor -- start 2024-09-19T09:33:52.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d84696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8465fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41ac740 con 0x5581d844e170 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41ac070 con 0x5581d844d850 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11b41a92f0 con 0x5581d8458a30 2024-09-19T09:33:52.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 10 client.0 _ll_register_callbacks cb 0x7f119c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:52.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48796/0 (socket says 172.21.15.28:48796) 2024-09-19T09:33:52.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3095427207 learned_addr learned my addr 172.21.15.28:0/3095427207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:52.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b259c640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d84696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b2d9d640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8465fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 msgr2=0x5581d8465fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8465fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3095427207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 msgr2=0x5581d84696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d84696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 -- 172.21.15.28:0/3095427207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d84664e0 con 0x5581d844e170 2024-09-19T09:33:52.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b259c640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d84696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11b1d9b640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c06e0 tx=0x7f119c0c4a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db53e1de52d79748 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f1199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c049070 con 0x5581d844e170 2024-09-19T09:33:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f119c0c0300 con 0x5581d844e170 2024-09-19T09:33:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.943+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a4009d40 con 0x5581d844e170 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f119c0044e0 con 0x5581d844e170 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581d84dc880 con 0x5581d844e170 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4009d40 con 0x5581d844e170 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f119c0bb640 con 0x5581d844e170 2024-09-19T09:33:52.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 10 client.15765 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 20 client.15765 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d84dce20 0x5581d84df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 4 client.15765 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5581d83b9be0 con 0x5581d84dce20 2024-09-19T09:33:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.944+0000 7f11b259c640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d84dce20 0x5581d84df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.945+0000 7f1199ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:52.945+0000 7f11b259c640 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d84dce20 0x5581d84df210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e82f285941d2fb server_cookie=292ef1106cedcdd0 in_seq=0 out_seq=0 2024-09-19T09:33:52.965 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/ 2024-09-19T09:33:52.966 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/m4/ 2024-09-19T09:33:52.967 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-19T09:33:52.968 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/man/ 2024-09-19T09:33:52.969 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/man/blogbench.8 2024-09-19T09:33:52.970 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/man/Makefile.am 2024-09-19T09:33:52.971 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/man/Makefile.in 2024-09-19T09:33:52.972 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/ 2024-09-19T09:33:52.973 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-19T09:33:52.973 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-19T09:33:52.974 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/blogbench.c 2024-09-19T09:33:52.975 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/blogbench.h 2024-09-19T09:33:52.976 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-19T09:33:52.977 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-19T09:33:52.978 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-19T09:33:52.979 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-19T09:33:52.980 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-19T09:33:52.981 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/Makefile.am 2024-09-19T09:33:52.982 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/Makefile.in 2024-09-19T09:33:52.983 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/process.c 2024-09-19T09:33:52.984 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/globals.h 2024-09-19T09:33:52.985 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/commenter.c 2024-09-19T09:33:52.986 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/rewriter.c 2024-09-19T09:33:52.987 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/reader.c 2024-09-19T09:33:52.988 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/writer.c 2024-09-19T09:33:52.989 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/src/helpers.c 2024-09-19T09:33:52.990 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/NEWS 2024-09-19T09:33:52.991 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/compile 2024-09-19T09:33:52.992 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/depcomp 2024-09-19T09:33:52.993 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/aclocal.m4 2024-09-19T09:33:52.994 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/README 2024-09-19T09:33:52.995 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/configure 2024-09-19T09:33:52.998 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/configure.ac 2024-09-19T09:33:52.999 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/config.guess 2024-09-19T09:33:53.001 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/install-sh 2024-09-19T09:33:53.002 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/config.sub 2024-09-19T09:33:53.003 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/missing 2024-09-19T09:33:53.004 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/mkinstalldirs 2024-09-19T09:33:53.005 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/Makefile.am 2024-09-19T09:33:53.006 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/Makefile.in 2024-09-19T09:33:53.007 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/config.h.in 2024-09-19T09:33:53.008 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/AUTHORS 2024-09-19T09:33:53.009 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/INSTALL 2024-09-19T09:33:53.010 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/ChangeLog 2024-09-19T09:33:53.011 INFO:tasks.workunit.client.0.smithi028.stdout:blogbench-1.0/COPYING 2024-09-19T09:33:53.012 INFO:tasks.workunit.client.0.smithi028.stderr:+ cd blogbench-1.0/ 2024-09-19T09:33:53.013 INFO:tasks.workunit.client.0.smithi028.stdout:making blogbench 2024-09-19T09:33:53.013 INFO:tasks.workunit.client.0.smithi028.stderr:+ echo 'making blogbench' 2024-09-19T09:33:53.013 INFO:tasks.workunit.client.0.smithi028.stderr:+ ./configure 2024-09-19T09:33:53.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.176+0000 7f1199ffb640 1 -- 172.21.15.28:0/3095427207 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5581d83b9be0 con 0x5581d84dce20 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.176+0000 7f1199ffb640 10 client.15765 handle_command_reply: tid=0 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.176+0000 7f11bb2c1740 1 client.15765 shutdown 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.176+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 msgr2=0x5581d8465a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.176+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c06e0 tx=0x7f119c0c4a80 comp rx=0 tx=0).stop 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 shutdown_connections 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581d84dce20 0x5581d84df210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581d8458a30 0x5581d8465fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581d844e170 0x5581d8465a60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f1199ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 --2- 172.21.15.28:0/3095427207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581d844d850 0x5581d84696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 >> 172.21.15.28:0/3095427207 conn(0x5581d8449de0 msgr2=0x5581d84486a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 shutdown_connections 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 1 -- 172.21.15.28:0/3095427207 wait complete. 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11bb2c1740 20 client.15765 trim_cache size 0 max 16384 2024-09-19T09:33:53.179 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:52 smithi028 ceph-mon[30972]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:33:53.179 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 6, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "5bd3b1f": { 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 6, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 140.3, 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:33:53.180 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:33:53.181 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 msgr2=0x7f11a00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 0x7f11a00a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005fd0 tx=0x7f11ac004a40 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 msgr2=0x7f11b419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 shutdown_connections 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f11a00a32c0 0x7f11a00a5780 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11b4106f10 0x7f11b419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11b4102790 0x7f11b419ab90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.177+0000 7f11b8ee8640 1 --2- 172.21.15.28:0/4186622055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b4101de0 0x7f11b419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.178+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 >> 172.21.15.28:0/4186622055 conn(0x7f11b40fd8c0 msgr2=0x7f11b40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:53.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.178+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 shutdown_connections 2024-09-19T09:33:53.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:33:53.178+0000 7f11b8ee8640 1 -- 172.21.15.28:0/4186622055 wait complete. 2024-09-19T09:33:53.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:52 smithi078 ceph-mon[35619]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:33:53.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:52 smithi038 ceph-mon[35271]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:33:54.047 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5bd3b1f', quiesce took 4.0 seconds. Will release after: 16.35 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 -- 172.21.15.28:0/883431331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c801025f0 msgr2=0x7f3c80108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 --2- 172.21.15.28:0/883431331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c801025f0 0x7f3c80108e80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009ca0 tx=0x7f3c7802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 -- 172.21.15.28:0/883431331 shutdown_connections 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 --2- 172.21.15.28:0/883431331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c801025f0 0x7f3c80108e80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 --2- 172.21.15.28:0/883431331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c80101c30 0x7f3c801020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 --2- 172.21.15.28:0/883431331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c80100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 -- 172.21.15.28:0/883431331 >> 172.21.15.28:0/883431331 conn(0x7f3c800fc1c0 msgr2=0x7f3c800fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 -- 172.21.15.28:0/883431331 shutdown_connections 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 -- 172.21.15.28:0/883431331 wait complete. 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.915+0000 7f3c85c82640 1 Processor -- start 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 -- start start 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c8006a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 0x7f3c8019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010c6c0 con 0x7f3c80101c30 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010c540 con 0x7f3c80100a40 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c85c82640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010c840 con 0x7f3c801025f0 2024-09-19T09:33:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59642/0 (socket says 172.21.15.28:59642) 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 -- 172.21.15.28:0/995621154 learned_addr learned my addr 172.21.15.28:0/995621154 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c8006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 0x7f3c8019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 -- 172.21.15.28:0/995621154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 msgr2=0x7f3c8019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 0x7f3c8019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 -- 172.21.15.28:0/995621154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 msgr2=0x7f3c8006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c8006a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 -- 172.21.15.28:0/995621154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8019faa0 con 0x7f3c80101c30 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c8006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 0x7f3c8019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.916+0000 7f3c7effd640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400d770 tx=0x7f3c7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.917+0000 7f3c7cff9640 1 -- 172.21.15.28:0/995621154 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74017070 con 0x7f3c80101c30 2024-09-19T09:33:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.917+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8019fd30 con 0x7f3c80101c30 2024-09-19T09:33:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.917+0000 7f3c7cff9640 1 -- 172.21.15.28:0/995621154 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74005500 con 0x7f3c80101c30 2024-09-19T09:33:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.917+0000 7f3c7cff9640 1 -- 172.21.15.28:0/995621154 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004e00 con 0x7f3c80101c30 2024-09-19T09:33:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.917+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c801a01d0 con 0x7f3c80101c30 2024-09-19T09:33:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c7cff9640 1 -- 172.21.15.28:0/995621154 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3c74004fa0 con 0x7f3c80101c30 2024-09-19T09:33:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 Processor -- start 2024-09-19T09:33:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 -- start start 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e87701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e8777c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.919+0000 7f3c88a21740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5592e86cf2f0 con 0x5592e87706f0 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c88a21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592e8677270 con 0x5592e8775780 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c88a21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592e86cb2b0 con 0x5592e876fdb0 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7cff9640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 0x7f3c680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59644/0 (socket says 172.21.15.28:59644) 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 -- 172.21.15.28:0/177636349 learned_addr learned my addr 172.21.15.28:0/177636349 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 0x7f3c680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e87701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7cff9640 1 -- 172.21.15.28:0/995621154 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3c740cf470 con 0x7f3c80101c30 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7effd640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e8777c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 -- 172.21.15.28:0/177636349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 msgr2=0x5592e87701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e87701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 -- 172.21.15.28:0/177636349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 msgr2=0x5592e8777c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e8777c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 -- 172.21.15.28:0/177636349 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e8676340 con 0x5592e87706f0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e87701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009cd0 tx=0x7f3c700117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43041e317aa55a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 0x7f3c680a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c78006010 tx=0x7f3c78031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c65ffb640 1 -- 172.21.15.28:0/177636349 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70012450 con 0x5592e87706f0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c65ffb640 1 -- 172.21.15.28:0/177636349 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70012ad0 con 0x5592e87706f0 2024-09-19T09:33:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.920+0000 7f3c65ffb640 1 -- 172.21.15.28:0/177636349 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000fc20 con 0x5592e87706f0 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- 172.21.15.28:0/177636349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 msgr2=0x5592e87751b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009cd0 tx=0x7f3c700117b0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- 172.21.15.28:0/177636349 shutdown_connections 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e8777c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e87751b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 --2- 172.21.15.28:0/177636349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e87701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- 172.21.15.28:0/177636349 >> 172.21.15.28:0/177636349 conn(0x5592e8767010 msgr2=0x5592e8767410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- 172.21.15.28:0/177636349 shutdown_connections 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- 172.21.15.28:0/177636349 wait complete. 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.921+0000 7f3c88a21740 1 -- start start 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 Processor -- start 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8779f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e87814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5592e8778750 con 0x5592e87706f0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592e8658290 con 0x5592e8775780 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592e87788d0 con 0x5592e876fdb0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c88a21740 10 client.0 _ll_register_callbacks cb 0x5592e8689140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59650/0 (socket says 172.21.15.28:59650) 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 -- 172.21.15.28:0/4283671237 learned_addr learned my addr 172.21.15.28:0/4283671237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e87814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8779f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 -- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 msgr2=0x5592e8779f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8779f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 -- 172.21.15.28:0/4283671237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 msgr2=0x5592e87814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e87814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7effd640 1 -- 172.21.15.28:0/4283671237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e83e2210 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.922+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8779f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c7effd640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c74097ec0 tx=0x7f3c74004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c97003b88c4e2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740b4060 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3c74009cc0 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c70012ad0 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c74020040 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 10 client.15777 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e84ae620 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70012ad0 con 0x5592e87706f0 2024-09-19T09:33:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c740980d0 con 0x5592e87706f0 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 20 client.15777 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e8785b60 0x5592e8787fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 4 client.15777 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.923+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e86dca20 con 0x5592e8785b60 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.924+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e8785b60 0x5592e8787fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.924+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e8785b60 0x5592e8787fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3bf4af778b39e7 server_cookie=3562e75b0afb2b5d in_seq=0 out_seq=0 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.924+0000 7f3c5effd640 10 client.15777 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c5effd640 1 -- 172.21.15.28:0/4283671237 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592e86dca20 con 0x5592e8785b60 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c5effd640 10 client.15777 handle_command_reply: tid=0 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 client.15777 shutdown 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 msgr2=0x5592e877a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c74097ec0 tx=0x7f3c74004780 comp rx=0 tx=0).stop 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 shutdown_connections 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e8775780 0x5592e87814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e8785b60 0x5592e8787fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e87706f0 0x5592e877a4b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 --2- 172.21.15.28:0/4283671237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8779f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c5effd640 0 client.15777 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 >> 172.21.15.28:0/4283671237 conn(0x5592e8767010 msgr2=0x5592e8768820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 shutdown_connections 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 1 -- 172.21.15.28:0/4283671237 wait complete. 2024-09-19T09:33:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.925+0000 7f3c88a21740 20 client.15777 trim_cache size 0 max 16384 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.939+0000 7f3c88a21740 1 Processor -- start 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.939+0000 7f3c88a21740 1 -- start start 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8778cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e87814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a6d20 con 0x5592e8775780 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a8860 con 0x5592e87706f0 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c88a21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a73f0 con 0x5592e876fdb0 2024-09-19T09:33:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59652/0 (socket says 172.21.15.28:59652) 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 -- 172.21.15.28:0/2027580142 learned_addr learned my addr 172.21.15.28:0/2027580142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8778cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7effd640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e87814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 -- 172.21.15.28:0/2027580142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 msgr2=0x5592e8778cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8778cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 -- 172.21.15.28:0/2027580142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 msgr2=0x5592e87814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e87814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 -- 172.21.15.28:0/2027580142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e8770bd0 con 0x5592e8775780 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7effd640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e87814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c78031e30 tx=0x7f3c78041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780743d3e9d0daa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c657fa640 1 -- 172.21.15.28:0/2027580142 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78044070 con 0x5592e8775780 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c657fa640 1 -- 172.21.15.28:0/2027580142 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c78036040 con 0x5592e8775780 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.940+0000 7f3c657fa640 1 -- 172.21.15.28:0/2027580142 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78030800 con 0x5592e8775780 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- 172.21.15.28:0/2027580142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 msgr2=0x5592e8767ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c78031e30 tx=0x7f3c78041000 comp rx=0 tx=0).stop 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- 172.21.15.28:0/2027580142 shutdown_connections 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e87814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e8767ec0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 --2- 172.21.15.28:0/2027580142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e8778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- 172.21.15.28:0/2027580142 >> 172.21.15.28:0/2027580142 conn(0x5592e8767010 msgr2=0x5592e8779f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- 172.21.15.28:0/2027580142 shutdown_connections 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- 172.21.15.28:0/2027580142 wait complete. 2024-09-19T09:33:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 -- start start 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.941+0000 7f3c88a21740 1 Processor -- start 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e877e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e877ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a8190 con 0x5592e8775780 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a7ac0 con 0x5592e87706f0 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c801a4d00 con 0x5592e876fdb0 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c88a21740 10 client.0 _ll_register_callbacks cb 0x7f3c74097d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59656/0 (socket says 172.21.15.28:59656) 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 -- 172.21.15.28:0/1919177609 learned_addr learned my addr 172.21.15.28:0/1919177609 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7effd640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e877ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e877e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 -- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 msgr2=0x5592e877e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e877e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 -- 172.21.15.28:0/1919177609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 msgr2=0x5592e877ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e877ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 -- 172.21.15.28:0/1919177609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e8780450 con 0x5592e8775780 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e877e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c7ffff640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c78004af0 tx=0x7f3c7802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc0ace5682dd72c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c5effd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.942+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78058070 con 0x5592e8775780 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c78036040 con 0x5592e8775780 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3c780305c0 con 0x5592e8775780 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c780307c0 con 0x5592e8775780 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-09-19T09:33:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e8781790 con 0x5592e8775780 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c78036040 con 0x5592e8775780 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c7803c410 con 0x5592e8775780 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 20 client.15783 populate_metadata read hostname 'smithi028' 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e87f0f80 0x5592e87f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 4 client.15783 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592e86d6d70 con 0x5592e87f0f80 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.943+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e87f0f80 0x5592e87f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.944+0000 7f3c5effd640 10 client.15783 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.944+0000 7f3c7f7fe640 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e87f0f80 0x5592e87f33a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d0f0923b818b56 server_cookie=7f62b5fbd5aa59c1 in_seq=0 out_seq=0 2024-09-19T09:33:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c5effd640 1 -- 172.21.15.28:0/1919177609 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592e86d6d70 con 0x5592e87f0f80 2024-09-19T09:33:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c5effd640 10 client.15783 handle_command_reply: tid=0 2024-09-19T09:33:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 client.15783 shutdown 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 msgr2=0x5592e877fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c78004af0 tx=0x7f3c7802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 shutdown_connections 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592e87706f0 0x5592e877ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5592e87f0f80 0x5592e87f33a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5592e8775780 0x5592e877fe40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 --2- 172.21.15.28:0/1919177609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592e876fdb0 0x5592e877e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c5effd640 0 client.15783 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 >> 172.21.15.28:0/1919177609 conn(0x5592e8767010 msgr2=0x5592e87659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 shutdown_connections 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 1 -- 172.21.15.28:0/1919177609 wait complete. 2024-09-19T09:33:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.968+0000 7f3c88a21740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T09:33:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:33:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:33:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:33:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 msgr2=0x7f3c680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 0x7f3c680a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3c78006010 tx=0x7f3c78031bb0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 msgr2=0x7f3c8019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400d770 tx=0x7f3c7400dc30 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 shutdown_connections 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3c680a3240 0x7f3c680a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025f0 0x7f3c8019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c80101c30 0x7f3c8019cb70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 --2- 172.21.15.28:0/995621154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c80100a40 0x7f3c8006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:33:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 >> 172.21.15.28:0/995621154 conn(0x7f3c800fc1c0 msgr2=0x7f3c80105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:33:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 shutdown_connections 2024-09-19T09:33:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:33:54.969+0000 7f3c85c82640 1 -- 172.21.15.28:0/995621154 wait complete. 2024-09-19T09:33:55.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:54 smithi028 ceph-mon[30972]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:33:55.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:54 smithi078 ceph-mon[35619]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:33:55.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:54 smithi038 ceph-mon[35271]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:33:57.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:57 smithi028 ceph-mon[30972]: pgmap v376: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T09:33:57.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:57 smithi078 ceph-mon[35619]: pgmap v376: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T09:33:57.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:57 smithi038 ceph-mon[35271]: pgmap v376: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T09:33:59.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:33:59 smithi028 ceph-mon[30972]: pgmap v377: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-19T09:33:59.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:33:59 smithi078 ceph-mon[35619]: pgmap v377: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-19T09:33:59.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:33:59 smithi038 ceph-mon[35271]: pgmap v377: 97 pgs: 97 active+clean; 900 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-19T09:34:01.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:01 smithi028 ceph-mon[30972]: pgmap v378: 97 pgs: 97 active+clean; 1.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-19T09:34:01.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:01 smithi078 ceph-mon[35619]: pgmap v378: 97 pgs: 97 active+clean; 1.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-19T09:34:01.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:01 smithi038 ceph-mon[35271]: pgmap v378: 97 pgs: 97 active+clean; 1.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 30 op/s 2024-09-19T09:34:02.092 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:34:02.365 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 -- 172.21.15.28:0/4156115316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 msgr2=0x7f2f9410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/4156115316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f9410bb50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009900 tx=0x7f2f8802a480 comp rx=0 tx=0).stop 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 -- 172.21.15.28:0/4156115316 shutdown_connections 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/4156115316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/4156115316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f9410bb50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/4156115316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 0x7f2f94103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 -- 172.21.15.28:0/4156115316 >> 172.21.15.28:0/4156115316 conn(0x7f2f940fc7a0 msgr2=0x7f2f940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 -- 172.21.15.28:0/4156115316 shutdown_connections 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.804+0000 7f2f9a18f640 1 -- 172.21.15.28:0/4156115316 wait complete. 2024-09-19T09:34:02.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.805+0000 7f2f9a18f640 1 Processor -- start 2024-09-19T09:34:02.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.805+0000 7f2f9a18f640 1 -- start start 2024-09-19T09:34:02.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 0x7f2f94198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94111fb0 con 0x7f2f94103660 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94111e30 con 0x7f2f9410c170 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f9a18f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94112130 con 0x7f2f94102d40 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59662/0 (socket says 172.21.15.28:59662) 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2440948197 learned_addr learned my addr 172.21.15.28:0/2440948197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 0x7f2f94198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2440948197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 msgr2=0x7f2f94198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 0x7f2f94198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2440948197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 msgr2=0x7f2f9419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2440948197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9419d410 con 0x7f2f94103660 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.806+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005ab0 tx=0x7f2f880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.807+0000 7f2f90ff9640 1 -- 172.21.15.28:0/2440948197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8803f030 con 0x7f2f94103660 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.807+0000 7f2f90ff9640 1 -- 172.21.15.28:0/2440948197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f88003ce0 con 0x7f2f94103660 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.807+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9419d6a0 con 0x7f2f94103660 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.807+0000 7f2f90ff9640 1 -- 172.21.15.28:0/2440948197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88035070 con 0x7f2f94103660 2024-09-19T09:34:02.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.807+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f941a4fe0 con 0x7f2f94103660 2024-09-19T09:34:02.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.809+0000 7f2f90ff9640 1 -- 172.21.15.28:0/2440948197 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2f88042030 con 0x7f2f94103660 2024-09-19T09:34:02.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.809+0000 7f2f9c568740 1 Processor -- start 2024-09-19T09:34:02.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.809+0000 7f2f9c568740 1 -- start start 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5c2dc0 0x55f01f5cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f90ff9640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 0x7f2f7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f01f5282e0 con 0x55f01f5cd680 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f01f4cfce0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f9c568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f01f5242a0 con 0x55f01f5c24a0 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 0x7f2f7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59666/0 (socket says 172.21.15.28:59666) 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2463051024 learned_addr learned my addr 172.21.15.28:0/2463051024 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f90ff9640 1 -- 172.21.15.28:0/2440948197 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2f880f0100 con 0x7f2f94103660 2024-09-19T09:34:02.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5c2dc0 0x55f01f5cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2463051024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 msgr2=0x55f01f5c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2463051024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5c2dc0 msgr2=0x55f01f5cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5c2dc0 0x55f01f5cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2463051024 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01f4b4440 con 0x55f01f5cd680 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 0x7f2f7c0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0126e0 tx=0x7f2f8c012ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.810+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2f880b8bb0 tx=0x7f2f88005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78646c0e3058029 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f79ffb640 1 -- 172.21.15.28:0/2463051024 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88049070 con 0x55f01f5cd680 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f79ffb640 1 -- 172.21.15.28:0/2463051024 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f880bce50 con 0x55f01f5cd680 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f79ffb640 1 -- 172.21.15.28:0/2463051024 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88003800 con 0x55f01f5cd680 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 -- 172.21.15.28:0/2463051024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 msgr2=0x55f01f5cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2f880b8bb0 tx=0x7f2f88005d50 comp rx=0 tx=0).stop 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 -- 172.21.15.28:0/2463051024 shutdown_connections 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5c2dc0 0x55f01f5cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5cd680 0x55f01f5cfa70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2463051024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 -- 172.21.15.28:0/2463051024 >> 172.21.15.28:0/2463051024 conn(0x55f01f5bea30 msgr2=0x55f01f5bee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.811+0000 7f2f9c568740 1 -- 172.21.15.28:0/2463051024 shutdown_connections 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 -- 172.21.15.28:0/2463051024 wait complete. 2024-09-19T09:34:02.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 -- start start 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 Processor -- start 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f01f5d0010 con 0x55f01f5c2dc0 2024-09-19T09:34:02.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f01f4e3050 con 0x55f01f5cd680 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f01f5d0190 con 0x55f01f5c24a0 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f9c568740 10 client.0 _ll_register_callbacks cb 0x55f01f523160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.812+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59680/0 (socket says 172.21.15.28:59680) 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2707263861 learned_addr learned my addr 172.21.15.28:0/2707263861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f93fff640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 msgr2=0x55f01f5cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2707263861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 msgr2=0x55f01f5ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 -- 172.21.15.28:0/2707263861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01f2993c0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2f8803d040 tx=0x7f2f8803b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56d4cdc9fefb3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8808b020 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2f8808b1c0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f880bce50 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f88049070 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f9c568740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.813+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f01f5d0010 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f880bce50 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2f8803bc90 con 0x55f01f5c2dc0 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:02.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 20 client.15795 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f5dcc50 0x55f01f5df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 4 client.15795 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f01f535850 con 0x55f01f5dcc50 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f5dcc50 0x55f01f5df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f937fe640 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f5dcc50 0x55f01f5df040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7b6df1e2f2588d server_cookie=b621965955a80a62 in_seq=0 out_seq=0 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.814+0000 7f2f72ffd640 10 client.15795 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f72ffd640 1 -- 172.21.15.28:0/2707263861 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f01f535850 con 0x55f01f5dcc50 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f72ffd640 10 client.15795 handle_command_reply: tid=0 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 client.15795 shutdown 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 msgr2=0x55f01f5ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2f8803d040 tx=0x7f2f8803b690 comp rx=0 tx=0).stop 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 shutdown_connections 2024-09-19T09:34:02.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f5dcc50 0x55f01f5df040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5ca2b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 --2- 172.21.15.28:0/2707263861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.815+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 >> 172.21.15.28:0/2707263861 conn(0x55f01f5bea30 msgr2=0x55f01f5c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.816+0000 7f2f72ffd640 0 client.15795 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.816+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 shutdown_connections 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.816+0000 7f2f9c568740 1 -- 172.21.15.28:0/2707263861 wait complete. 2024-09-19T09:34:02.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.816+0000 7f2f9c568740 20 client.15795 trim_cache size 0 max 16384 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 Processor -- start 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 -- start start 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c2dc0 0x55f01f5e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941ad470 con 0x55f01f5c24a0 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941aefb0 con 0x55f01f5cd680 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f9c568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941adb40 con 0x55f01f5c2dc0 2024-09-19T09:34:02.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59694/0 (socket says 172.21.15.28:59694) 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f937fe640 1 -- 172.21.15.28:0/1484037224 learned_addr learned my addr 172.21.15.28:0/1484037224 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.833+0000 7f2f93fff640 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c2dc0 0x55f01f5e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 -- 172.21.15.28:0/1484037224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c2dc0 msgr2=0x55f01f5e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c2dc0 0x55f01f5e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 -- 172.21.15.28:0/1484037224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 msgr2=0x55f01f5e4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5e4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 -- 172.21.15.28:0/1484037224 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01f5c2bf0 con 0x55f01f5c24a0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f937fe640 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2f84000c00 tx=0x7f2f84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e341464617eb9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f797fa640 1 -- 172.21.15.28:0/1484037224 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84017e40 con 0x55f01f5c24a0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f797fa640 1 -- 172.21.15.28:0/1484037224 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f8401d730 con 0x55f01f5c24a0 2024-09-19T09:34:02.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.834+0000 7f2f797fa640 1 -- 172.21.15.28:0/1484037224 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84015780 con 0x55f01f5c24a0 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- 172.21.15.28:0/1484037224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 msgr2=0x55f01f5d91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2f84000c00 tx=0x7f2f84008040 comp rx=0 tx=0).stop 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- 172.21.15.28:0/1484037224 shutdown_connections 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c2dc0 0x55f01f5e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 --2- 172.21.15.28:0/1484037224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c24a0 0x55f01f5d91e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- 172.21.15.28:0/1484037224 >> 172.21.15.28:0/1484037224 conn(0x55f01f5bea30 msgr2=0x55f01f5c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- 172.21.15.28:0/1484037224 shutdown_connections 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- 172.21.15.28:0/1484037224 wait complete. 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 -- start start 2024-09-19T09:34:02.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 Processor -- start 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.835+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941ae8e0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941ae210 con 0x55f01f5cd680 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941ab450 con 0x55f01f5c24a0 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 10 client.0 _ll_register_callbacks cb 0x7f2f840093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59704/0 (socket says 172.21.15.28:59704) 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 -- 172.21.15.28:0/3270889757 learned_addr learned my addr 172.21.15.28:0/3270889757 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f93fff640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f937fe640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 -- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 msgr2=0x55f01f5d5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5d5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 -- 172.21.15.28:0/3270889757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 msgr2=0x55f01f5d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 -- 172.21.15.28:0/3270889757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01f5d15e0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f93fff640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f937fe640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f92ffd640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f88004c60 tx=0x7f2f88005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cd6258bcb366b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880bce50 con 0x55f01f5c2dc0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2f880baa70 con 0x55f01f5c2dc0 2024-09-19T09:34:02.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.836+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f8401d730 con 0x55f01f5c2dc0 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f880c0b00 con 0x55f01f5c2dc0 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f01f5d15e0 con 0x55f01f5c2dc0 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f8401d730 con 0x55f01f5c2dc0 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2f880c4f30 con 0x55f01f5c2dc0 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 20 client.15801 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f64be20 0x55f01f64e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 4 client.15801 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f01f52fd60 con 0x55f01f64be20 2024-09-19T09:34:02.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.837+0000 7f2f937fe640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f64be20 0x55f01f64e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f937fe640 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f64be20 0x55f01f64e240 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6dee9e22e536e1 server_cookie=3e7947d41ed1444f in_seq=0 out_seq=0 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f72ffd640 10 client.15801 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f72ffd640 1 -- 172.21.15.28:0/3270889757 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f01f52fd60 con 0x55f01f64be20 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f72ffd640 10 client.15801 handle_command_reply: tid=0 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 client.15801 shutdown 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 msgr2=0x55f01f5d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f88004c60 tx=0x7f2f88005b70 comp rx=0 tx=0).stop 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 shutdown_connections 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f01f5cd680 0x55f01f5d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55f01f64be20 0x55f01f64e240 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f01f5c2dc0 0x55f01f5d0b30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 --2- 172.21.15.28:0/3270889757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01f5c24a0 0x55f01f5d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 >> 172.21.15.28:0/3270889757 conn(0x55f01f5bea30 msgr2=0x55f01f5c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f72ffd640 0 client.15801 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 shutdown_connections 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 1 -- 172.21.15.28:0/3270889757 wait complete. 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.838+0000 7f2f9c568740 20 client.15801 trim_cache size 0 max 16384 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2024-09-19T09:34:02.847 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 msgr2=0x7f2f7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 0x7f2f7c0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0126e0 tx=0x7f2f8c012ae0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 msgr2=0x7f2f941989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005ab0 tx=0x7f2f880042f0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 shutdown_connections 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2f7c0a3240 0x7f2f7c0a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f9410c170 0x7f2f9419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f94103660 0x7f2f941989a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 --2- 172.21.15.28:0/2440948197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f94102d40 0x7f2f94198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 >> 172.21.15.28:0/2440948197 conn(0x7f2f940fc7a0 msgr2=0x7f2f940fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 shutdown_connections 2024-09-19T09:34:02.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:02.839+0000 7f2f9a18f640 1 -- 172.21.15.28:0/2440948197 wait complete. 2024-09-19T09:34:03.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:03 smithi028 ceph-mon[30972]: pgmap v379: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T09:34:03.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:03 smithi078 ceph-mon[35619]: pgmap v379: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T09:34:03.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:03 smithi038 ceph-mon[35271]: pgmap v379: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T09:34:03.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:34:03.560 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:34:03.819 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:04.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 -- 172.21.15.28:0/3286305909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4810e3c0 msgr2=0x7faa48110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:04.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 --2- 172.21.15.28:0/3286305909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4810e3c0 0x7faa48110810 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faa40009ca0 tx=0x7faa4002a950 comp rx=0 tx=0).stop 2024-09-19T09:34:04.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 -- 172.21.15.28:0/3286305909 shutdown_connections 2024-09-19T09:34:04.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 --2- 172.21.15.28:0/3286305909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4810e3c0 0x7faa48110810 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 --2- 172.21.15.28:0/3286305909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa48074f40 0x7faa4810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 --2- 172.21.15.28:0/3286305909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074620 0x7faa48074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 -- 172.21.15.28:0/3286305909 >> 172.21.15.28:0/3286305909 conn(0x7faa480fd880 msgr2=0x7faa480ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 -- 172.21.15.28:0/3286305909 shutdown_connections 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.218+0000 7faa4f260640 1 -- 172.21.15.28:0/3286305909 wait complete. 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 Processor -- start 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 -- start start 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074f40 0x7faa4819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa4810e3c0 0x7faa4819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa48114150 con 0x7faa48074620 2024-09-19T09:34:04.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa48113fd0 con 0x7faa48074f40 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4f260640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa481142d0 con 0x7faa4810e3c0 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35830/0 (socket says 172.21.15.28:35830) 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 -- 172.21.15.28:0/4010010635 learned_addr learned my addr 172.21.15.28:0/4010010635 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4d7fd640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa4810e3c0 0x7faa4819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa47fff640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074f40 0x7faa4819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 -- 172.21.15.28:0/4010010635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa4810e3c0 msgr2=0x7faa4819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa4810e3c0 0x7faa4819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 -- 172.21.15.28:0/4010010635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074f40 msgr2=0x7faa4819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074f40 0x7faa4819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.219+0000 7faa4cffc640 1 -- 172.21.15.28:0/4010010635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4819f5a0 con 0x7faa48074620 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa4cffc640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa3800d950 tx=0x7faa3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:04.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38017070 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa38009c30 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3800f6b0 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4819f890 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.220+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a7170 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.221+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa48074a00 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.222+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7faa380109d0 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.223+0000 7faa45ffb640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 0x7faa340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.223+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7faa380cf420 con 0x7faa48074620 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.223+0000 7faa47fff640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 0x7faa340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.224+0000 7faa47fff640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 0x7faa340a5780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa4810f6e0 tx=0x7faa3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:04.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.227+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faa38009de0 con 0x7faa48074620 2024-09-19T09:34:04.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.444+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa4819b950 con 0x7faa48074620 2024-09-19T09:34:04.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.444+0000 7faa45ffb640 1 -- 172.21.15.28:0/4010010635 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7faa3809bb20 con 0x7faa48074620 2024-09-19T09:34:04.447 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:34:04.447 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:34:04.447 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:34:04.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.446+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 msgr2=0x7faa340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:04.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.446+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 0x7faa340a5780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa4810f6e0 tx=0x7faa3000b390 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.446+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 msgr2=0x7faa4819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.446+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa3800d950 tx=0x7faa3800de10 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.446+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 shutdown_connections 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7faa340a32c0 0x7faa340a5780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48074f40 0x7faa4819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa4810e3c0 0x7faa4819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 --2- 172.21.15.28:0/4010010635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa48074620 0x7faa4819a5f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 >> 172.21.15.28:0/4010010635 conn(0x7faa480fd880 msgr2=0x7faa480fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 shutdown_connections 2024-09-19T09:34:04.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:04.447+0000 7faa4f260640 1 -- 172.21.15.28:0/4010010635 wait complete. 2024-09-19T09:34:05.024 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:34:05.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:05 smithi028 ceph-mon[30972]: pgmap v380: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-09-19T09:34:05.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:05 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4010010635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:05.281 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:05.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:05 smithi078 ceph-mon[35619]: pgmap v380: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-09-19T09:34:05.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:05 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4010010635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:05.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:05 smithi038 ceph-mon[35271]: pgmap v380: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-09-19T09:34:05.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:05 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4010010635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 -- 172.21.15.28:0/4265183304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660100a80 msgr2=0x7fa660100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 --2- 172.21.15.28:0/4265183304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660100a80 0x7fa660100ee0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa64800fb40 tx=0x7fa64802e5b0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 -- 172.21.15.28:0/4265183304 shutdown_connections 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 --2- 172.21.15.28:0/4265183304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660101420 0x7fa66010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 --2- 172.21.15.28:0/4265183304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660100a80 0x7fa660100ee0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 --2- 172.21.15.28:0/4265183304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660106a80 0x7fa660106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 -- 172.21.15.28:0/4265183304 >> 172.21.15.28:0/4265183304 conn(0x7fa6600fc7a0 msgr2=0x7fa6600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 -- 172.21.15.28:0/4265183304 shutdown_connections 2024-09-19T09:34:05.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.730+0000 7fa66584c640 1 -- 172.21.15.28:0/4265183304 wait complete. 2024-09-19T09:34:05.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 Processor -- start 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 -- start start 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660100a80 0x7fa660196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660106a80 0x7fa66019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa66010fd70 con 0x7fa660101420 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa66010fbf0 con 0x7fa660106a80 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa66584c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa66010fef0 con 0x7fa660100a80 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35840/0 (socket says 172.21.15.28:35840) 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa6567fc640 1 -- 172.21.15.28:0/1181354810 learned_addr learned my addr 172.21.15.28:0/1181354810 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa65effd640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660100a80 0x7fa660196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.731+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660106a80 0x7fa66019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 -- 172.21.15.28:0/1181354810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660100a80 msgr2=0x7fa660196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660100a80 0x7fa660196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 -- 172.21.15.28:0/1181354810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660106a80 msgr2=0x7fa66019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660106a80 0x7fa66019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 -- 172.21.15.28:0/1181354810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66019b240 con 0x7fa660101420 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa6567fc640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa648005940 tx=0x7fa64800f340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa66484a640 1 -- 172.21.15.28:0/1181354810 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648044070 con 0x7fa660101420 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66019b4d0 con 0x7fa660101420 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa66484a640 1 -- 172.21.15.28:0/1181354810 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa648004020 con 0x7fa660101420 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa66484a640 1 -- 172.21.15.28:0/1181354810 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648003be0 con 0x7fa660101420 2024-09-19T09:34:05.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.732+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6601a2e10 con 0x7fa660101420 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa66484a640 1 -- 172.21.15.28:0/1181354810 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa64803e040 con 0x7fa660101420 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 Processor -- start 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 -- start start 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdaf4b40 0x559ecdaf3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdafe370 0x559ecdb00760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559ecda592f0 con 0x559ecdaf35d0 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ecda00cf0 con 0x559ecdaf4b40 2024-09-19T09:34:05.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa667c25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ecda552b0 con 0x559ecdafe370 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa65effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa65effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35844/0 (socket says 172.21.15.28:35844) 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa65effd640 1 -- 172.21.15.28:0/78698740 learned_addr learned my addr 172.21.15.28:0/78698740 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdaf4b40 0x559ecdaf3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa66484a640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.735+0000 7fa6567fc640 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdafe370 0x559ecdb00760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 -- 172.21.15.28:0/78698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdafe370 msgr2=0x559ecdb00760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdafe370 0x559ecdb00760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 -- 172.21.15.28:0/78698740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdaf4b40 msgr2=0x559ecdaf3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa66484a640 1 -- 172.21.15.28:0/1181354810 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa6480f38d0 con 0x7fa660101420 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdaf4b40 0x559ecdaf3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 -- 172.21.15.28:0/78698740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecd9e53a0 con 0x559ecdaf35d0 2024-09-19T09:34:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65effd640 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa65000db60 tx=0x7fa650005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d91065d022e3055 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa655ffb640 1 -- 172.21.15.28:0/78698740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65001d070 con 0x559ecdaf35d0 2024-09-19T09:34:05.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa655ffb640 1 -- 172.21.15.28:0/78698740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65000de50 con 0x559ecdaf35d0 2024-09-19T09:34:05.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 0x7fa64c0a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa658009fd0 tx=0x7fa658004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.736+0000 7fa655ffb640 1 -- 172.21.15.28:0/78698740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500186a0 con 0x559ecdaf35d0 2024-09-19T09:34:05.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- 172.21.15.28:0/78698740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 msgr2=0x559ecdaf3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa65000db60 tx=0x7fa650005f90 comp rx=0 tx=0).stop 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- 172.21.15.28:0/78698740 shutdown_connections 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdafe370 0x559ecdb00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaf3a10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 --2- 172.21.15.28:0/78698740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdaf4b40 0x559ecdaf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- 172.21.15.28:0/78698740 >> 172.21.15.28:0/78698740 conn(0x559ecdabc310 msgr2=0x559ecdaefc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- 172.21.15.28:0/78698740 shutdown_connections 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- 172.21.15.28:0/78698740 wait complete. 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.737+0000 7fa667c25740 1 -- start start 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 Processor -- start 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdafdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdafb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559ecdb01070 con 0x559ecdaf4b40 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ecda14060 con 0x559ecdafe370 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ecdb011f0 con 0x559ecdaf35d0 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa667c25740 10 client.0 _ll_register_callbacks cb 0x559ecd7678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35856/0 (socket says 172.21.15.28:35856) 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 -- 172.21.15.28:0/2706712478 learned_addr learned my addr 172.21.15.28:0/2706712478 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa65effd640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdafdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdafb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 -- 172.21.15.28:0/2706712478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 msgr2=0x559ecdafdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdafdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 -- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 msgr2=0x559ecdafb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdafb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa6567fc640 1 -- 172.21.15.28:0/2706712478 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecd7ca3c0 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdafb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.738+0000 7fa65effd640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdafdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa648009430 tx=0x7fa64808fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6f36a22b1bb4ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480bf880 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa6480041d0 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65000de50 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6480c0620 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ecdb0ca50 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65000de50 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa6480c3020 con 0x559ecdaf4b40 2024-09-19T09:34:05.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 10 client.15819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 20 client.15819 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb0deb0 0x559ecdb10300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 4 client.15819 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.739+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ecda66860 con 0x559ecdb0deb0 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.740+0000 7fa65effd640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb0deb0 0x559ecdb10300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.740+0000 7fa65effd640 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb0deb0 0x559ecdb10300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1becaaee2de1d2bd server_cookie=bdaf3929fee9d3e6 in_seq=0 out_seq=0 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.740+0000 7fa63effd640 10 client.15819 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa63effd640 1 -- 172.21.15.28:0/2706712478 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ecda66860 con 0x559ecdb0deb0 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa63effd640 10 client.15819 handle_command_reply: tid=0 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 client.15819 shutdown 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 msgr2=0x559ecdafaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa648009430 tx=0x7fa64808fb90 comp rx=0 tx=0).stop 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 shutdown_connections 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb0deb0 0x559ecdb10300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdafb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdafaff0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 --2- 172.21.15.28:0/2706712478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdafdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa63effd640 0 client.15819 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 >> 172.21.15.28:0/2706712478 conn(0x559ecdabc310 msgr2=0x559ecdaf5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 shutdown_connections 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 1 -- 172.21.15.28:0/2706712478 wait complete. 2024-09-19T09:34:05.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.741+0000 7fa667c25740 20 client.15819 trim_cache size 0 max 16384 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.754+0000 7fa667c25740 1 Processor -- start 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.754+0000 7fa667c25740 1 -- start start 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.754+0000 7fa667c25740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa667c25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf4b40 0x559ecdb0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa667c25740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdaf6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa667c25740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601ab350 con 0x559ecdaf35d0 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa667c25740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601ace90 con 0x559ecdafe370 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa667c25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601aba20 con 0x559ecdaf4b40 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35866/0 (socket says 172.21.15.28:35866) 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 -- 172.21.15.28:0/1635718592 learned_addr learned my addr 172.21.15.28:0/1635718592 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:05.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdaf6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa6567fc640 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf4b40 0x559ecdb0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 -- 172.21.15.28:0/1635718592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf4b40 msgr2=0x559ecdb0a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf4b40 0x559ecdb0a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 -- 172.21.15.28:0/1635718592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 msgr2=0x559ecdaf6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdaf6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 -- 172.21.15.28:0/1635718592 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecdafe0b0 con 0x559ecdaf35d0 2024-09-19T09:34:05.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa65effd640 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa65000db30 tx=0x7fa650003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d10f4f3f44206c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa6557fa640 1 -- 172.21.15.28:0/1635718592 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650029070 con 0x559ecdaf35d0 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa6557fa640 1 -- 172.21.15.28:0/1635718592 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65001e040 con 0x559ecdaf35d0 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.755+0000 7fa6557fa640 1 -- 172.21.15.28:0/1635718592 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65001d070 con 0x559ecdaf35d0 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- 172.21.15.28:0/1635718592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 msgr2=0x559ecdaefe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa65000db30 tx=0x7fa650003d30 comp rx=0 tx=0).stop 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- 172.21.15.28:0/1635718592 shutdown_connections 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdaf6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf4b40 0x559ecdb0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 --2- 172.21.15.28:0/1635718592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf35d0 0x559ecdaefe10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- 172.21.15.28:0/1635718592 >> 172.21.15.28:0/1635718592 conn(0x559ecdabc310 msgr2=0x559ecdaef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- 172.21.15.28:0/1635718592 shutdown_connections 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- 172.21.15.28:0/1635718592 wait complete. 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.756+0000 7fa667c25740 1 -- start start 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 Processor -- start 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdb02290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601ac7c0 con 0x559ecdaf4b40 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601ac0f0 con 0x559ecdafe370 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601a9330 con 0x559ecdaf35d0 2024-09-19T09:34:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa667c25740 10 client.0 _ll_register_callbacks cb 0x7fa648004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa6567fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35874/0 (socket says 172.21.15.28:35874) 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:58654/0 (socket says 172.21.15.28:58654) 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa6567fc640 1 -- 172.21.15.28:0/2469802883 learned_addr learned my addr 172.21.15.28:0/2469802883 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdb02290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 -- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 msgr2=0x559ecdb02290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdb02290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 -- 172.21.15.28:0/2469802883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 msgr2=0x559ecdb01d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.757+0000 7fa65effd640 1 -- 172.21.15.28:0/2469802883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecdb028d0 con 0x559ecdaf35d0 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa65f7fe640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdb02290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa65effd640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa65000d790 tx=0x7fa650003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d612a4a96d4d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa63effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65001d070 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65001e040 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa650003670 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa650038030 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa667c25740 10 client.25408 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ecdb789f0 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.758+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65001e040 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa65003b070 con 0x559ecdaf35d0 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:05.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 10 client.25408 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 20 client.25408 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb78ec0 0x559ecdb7b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 4 client.25408 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ecda60d70 con 0x559ecdb78ec0 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb78ec0 0x559ecdb7b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa63effd640 10 client.25408 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.759+0000 7fa6567fc640 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb78ec0 0x559ecdb7b2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d795f0528b551cf9 server_cookie=3327fb88bf468860 in_seq=0 out_seq=0 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa63effd640 1 -- 172.21.15.28:0/2469802883 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ecda60d70 con 0x559ecdb78ec0 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa63effd640 10 client.25408 handle_command_reply: tid=0 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 client.25408 shutdown 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 msgr2=0x559ecdb06a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa65000d790 tx=0x7fa650003c80 comp rx=0 tx=0).stop 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 shutdown_connections 2024-09-19T09:34:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x559ecdb78ec0 0x559ecdb7b2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ecdafe370 0x559ecdb02290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa63effd640 0 client.25408 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ecdaf4b40 0x559ecdb01d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 --2- 172.21.15.28:0/2469802883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ecdaf35d0 0x559ecdb06a20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 >> 172.21.15.28:0/2469802883 conn(0x559ecdabc310 msgr2=0x559ecdafd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 shutdown_connections 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 1 -- 172.21.15.28:0/2469802883 wait complete. 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.760+0000 7fa667c25740 20 client.25408 trim_cache size 0 max 16384 2024-09-19T09:34:05.767 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 msgr2=0x7fa64c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 0x7fa64c0a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa658009fd0 tx=0x7fa658004ab0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 msgr2=0x7fa6601967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa648005940 tx=0x7fa64800f340 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 shutdown_connections 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa660106a80 0x7fa66019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa660101420 0x7fa6601967d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 --2- 172.21.15.28:0/1181354810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa660100a80 0x7fa660196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 >> 172.21.15.28:0/1181354810 conn(0x7fa6600fc7a0 msgr2=0x7fa6600fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:05.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 shutdown_connections 2024-09-19T09:34:05.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:05.761+0000 7fa66584c640 1 -- 172.21.15.28:0/1181354810 wait complete. 2024-09-19T09:34:06.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:34:06.714 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:07.095 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:07 smithi028 ceph-mon[30972]: pgmap v381: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.177+0000 7fda41785640 1 -- 172.21.15.28:0/107002546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c106f10 msgr2=0x7fda3c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.177+0000 7fda41785640 1 --2- 172.21.15.28:0/107002546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c106f10 0x7fda3c10ea90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda34009900 tx=0x7fda3402a410 comp rx=0 tx=0).stop 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 -- 172.21.15.28:0/107002546 shutdown_connections 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 --2- 172.21.15.28:0/107002546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c106f10 0x7fda3c10ea90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 --2- 172.21.15.28:0/107002546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c102790 0x7fda3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 --2- 172.21.15.28:0/107002546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c101de0 0x7fda3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 -- 172.21.15.28:0/107002546 >> 172.21.15.28:0/107002546 conn(0x7fda3c0fd8c0 msgr2=0x7fda3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 -- 172.21.15.28:0/107002546 shutdown_connections 2024-09-19T09:34:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 -- 172.21.15.28:0/107002546 wait complete. 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 Processor -- start 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.178+0000 7fda41785640 1 -- start start 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c101de0 0x7fda3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c106f10 0x7fda3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c111fc0 con 0x7fda3c102790 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c111e40 con 0x7fda3c106f10 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda41785640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c112140 con 0x7fda3c101de0 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35886/0 (socket says 172.21.15.28:35886) 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 -- 172.21.15.28:0/3104340021 learned_addr learned my addr 172.21.15.28:0/3104340021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:07.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3affd640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c101de0 0x7fda3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c106f10 0x7fda3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 -- 172.21.15.28:0/3104340021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c101de0 msgr2=0x7fda3c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c101de0 0x7fda3c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 -- 172.21.15.28:0/3104340021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c106f10 msgr2=0x7fda3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c106f10 0x7fda3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 -- 172.21.15.28:0/3104340021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1a68d0 con 0x7fda3c102790 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fda3000c920 tx=0x7fda3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.179+0000 7fda23fff640 1 -- 172.21.15.28:0/3104340021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30007c70 con 0x7fda3c102790 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.180+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a6bc0 con 0x7fda3c102790 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.180+0000 7fda23fff640 1 -- 172.21.15.28:0/3104340021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda30007e10 con 0x7fda3c102790 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.180+0000 7fda23fff640 1 -- 172.21.15.28:0/3104340021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30018890 con 0x7fda3c102790 2024-09-19T09:34:07.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.180+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a7150 con 0x7fda3c102790 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.182+0000 7fda23fff640 1 -- 172.21.15.28:0/3104340021 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fda30018b30 con 0x7fda3c102790 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.182+0000 7fda43b5e740 1 Processor -- start 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.182+0000 7fda43b5e740 1 -- start start 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 0x55f5047ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047ae5d0 0x55f5047aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5047142f0 con 0x55f5047b9370 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f5046bbcf0 con 0x55f5047ae5d0 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda43b5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5047102b0 con 0x55f5047afb40 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35894/0 (socket says 172.21.15.28:35894) 2024-09-19T09:34:07.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 0x55f5047ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3a7fc640 1 -- 172.21.15.28:0/1351316490 learned_addr learned my addr 172.21.15.28:0/1351316490 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda23fff640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 0x7fda240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3affd640 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047ae5d0 0x55f5047aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1351316490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047ae5d0 msgr2=0x55f5047aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047ae5d0 0x55f5047aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1351316490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 msgr2=0x55f5047bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1351316490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5046a03a0 con 0x55f5047afb40 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda23fff640 1 -- 172.21.15.28:0/3104340021 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fda300cf900 con 0x7fda3c102790 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 0x7fda240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.183+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 0x55f5047ae090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fda3402a930 tx=0x7fda340041c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fdaf77962090ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda217fa640 1 -- 172.21.15.28:0/1351316490 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035040 con 0x55f5047afb40 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda217fa640 1 -- 172.21.15.28:0/1351316490 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda340315b0 con 0x55f5047afb40 2024-09-19T09:34:07.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda217fa640 1 -- 172.21.15.28:0/1351316490 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34046640 con 0x55f5047afb40 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 0x7fda240a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda34047000 tx=0x7fda34048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 -- 172.21.15.28:0/1351316490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 msgr2=0x55f5047ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 0x55f5047ae090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fda3402a930 tx=0x7fda340041c0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 -- 172.21.15.28:0/1351316490 shutdown_connections 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047ae5d0 0x55f5047aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1351316490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047afb40 0x55f5047ae090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 -- 172.21.15.28:0/1351316490 >> 172.21.15.28:0/1351316490 conn(0x55f504777310 msgr2=0x55f5047aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 -- 172.21.15.28:0/1351316490 shutdown_connections 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.184+0000 7fda43b5e740 1 -- 172.21.15.28:0/1351316490 wait complete. 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 -- start start 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 Processor -- start 2024-09-19T09:34:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5047bc070 con 0x55f5047b9370 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f5046cf060 con 0x55f5047afb40 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5047bc1f0 con 0x55f5047ae5d0 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda43b5e740 10 client.0 _ll_register_callbacks cb 0x55f5044228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35900/0 (socket says 172.21.15.28:35900) 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1078579789 learned_addr learned my addr 172.21.15.28:0/1078579789 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3affd640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 msgr2=0x55f5047b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1078579789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 msgr2=0x55f5047b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.185+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1078579789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5044853c0 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda340466a0 tx=0x7fda34031e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907fd9bb318398d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003be0 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda340315b0 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fda34031760 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda34063050 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda43b5e740 10 client.15837 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5047bc070 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda340315b0 con 0x55f5047b9370 2024-09-19T09:34:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fda34051480 con 0x55f5047b9370 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda43b5e740 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.186+0000 7fda43b5e740 10 client.15837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda43b5e740 20 client.15837 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f5047c8cc0 0x55f5047cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda43b5e740 4 client.15837 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f504721860 con 0x55f5047c8cc0 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda3affd640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f5047c8cc0 0x55f5047cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda1eff5640 10 client.15837 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.187+0000 7fda3affd640 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f5047c8cc0 0x55f5047cb0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f866e0bb72c0b9a2 server_cookie=b9bde072dac085f5 in_seq=0 out_seq=0 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda1eff5640 1 -- 172.21.15.28:0/1078579789 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f504721860 con 0x55f5047c8cc0 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda1eff5640 10 client.15837 handle_command_reply: tid=0 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 client.15837 shutdown 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 msgr2=0x55f5047b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda340466a0 tx=0x7fda34031e90 comp rx=0 tx=0).stop 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 shutdown_connections 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f5047c8cc0 0x55f5047cb0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047b6640 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda1eff5640 0 client.15837 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1078579789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 >> 172.21.15.28:0/1078579789 conn(0x55f504777310 msgr2=0x55f5047b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 shutdown_connections 2024-09-19T09:34:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 1 -- 172.21.15.28:0/1078579789 wait complete. 2024-09-19T09:34:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.188+0000 7fda43b5e740 20 client.15837 trim_cache size 0 max 16384 2024-09-19T09:34:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 Processor -- start 2024-09-19T09:34:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 -- start start 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1af540 con 0x55f5047b9370 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1b1080 con 0x55f5047afb40 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.201+0000 7fda43b5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1afc10 con 0x55f5047ae5d0 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35914/0 (socket says 172.21.15.28:35914) 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1532401366 learned_addr learned my addr 172.21.15.28:0/1532401366 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3affd640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1532401366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 msgr2=0x55f5047c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1532401366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 msgr2=0x55f5047d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 -- 172.21.15.28:0/1532401366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5047b4580 con 0x55f5047b9370 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda34046670 tx=0x7fda3402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff2a045c428856f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda20ff9640 1 -- 172.21.15.28:0/1532401366 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34051540 con 0x55f5047b9370 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda20ff9640 1 -- 172.21.15.28:0/1532401366 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda340040c0 con 0x55f5047b9370 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.202+0000 7fda20ff9640 1 -- 172.21.15.28:0/1532401366 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34046e40 con 0x55f5047b9370 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- 172.21.15.28:0/1532401366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 msgr2=0x55f5047d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda34046670 tx=0x7fda3402a410 comp rx=0 tx=0).stop 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- 172.21.15.28:0/1532401366 shutdown_connections 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047d0dd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 --2- 172.21.15.28:0/1532401366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- 172.21.15.28:0/1532401366 >> 172.21.15.28:0/1532401366 conn(0x55f504777310 msgr2=0x55f5047b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- 172.21.15.28:0/1532401366 shutdown_connections 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- 172.21.15.28:0/1532401366 wait complete. 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 -- start start 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.203+0000 7fda43b5e740 1 Processor -- start 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1b09b0 con 0x55f5047b9370 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1b02e0 con 0x55f5047afb40 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ad520 con 0x55f5047ae5d0 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda43b5e740 10 client.0 _ll_register_callbacks cb 0x7fda3404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35924/0 (socket says 172.21.15.28:35924) 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 -- 172.21.15.28:0/2861491502 learned_addr learned my addr 172.21.15.28:0/2861491502 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3affd640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3a7fc640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 -- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 msgr2=0x55f5047c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 -- 172.21.15.28:0/2861491502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 msgr2=0x55f5047bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 -- 172.21.15.28:0/2861491502 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5047bd810 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda3b7fe640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda34009850 tx=0x7fda340468a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4998e46083e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.204+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035070 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fda3404d070 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda340040c0 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda34051c20 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5048335b0 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda340040c0 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fda34031ad0 con 0x55f5047b9370 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:07.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 10 client.15843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 20 client.15843 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f504833b10 0x55f504835f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 4 client.15843 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f50471bd70 con 0x55f504833b10 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.205+0000 7fda3affd640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f504833b10 0x55f504835f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda3affd640 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f504833b10 0x55f504835f00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f4062ee7b3f1356 server_cookie=262d2981254bacbf in_seq=0 out_seq=0 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda1eff5640 10 client.15843 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda1eff5640 1 -- 172.21.15.28:0/2861491502 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f50471bd70 con 0x55f504833b10 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda1eff5640 10 client.15843 handle_command_reply: tid=0 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 client.15843 shutdown 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 msgr2=0x55f5047bd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda34009850 tx=0x7fda340468a0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 shutdown_connections 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f504833b10 0x55f504835f00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5047afb40 0x55f5047bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f5047b9370 0x55f5047bd220 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 --2- 172.21.15.28:0/2861491502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5047ae5d0 0x55f5047c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 >> 172.21.15.28:0/2861491502 conn(0x55f504777310 msgr2=0x55f5047a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda1eff5640 0 client.15843 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 shutdown_connections 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 1 -- 172.21.15.28:0/2861491502 wait complete. 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.206+0000 7fda43b5e740 20 client.15843 trim_cache size 0 max 16384 2024-09-19T09:34:07.218 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 msgr2=0x7fda240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 0x7fda240a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda34047000 tx=0x7fda34048000 comp rx=0 tx=0).stop 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 msgr2=0x7fda3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fda3000c920 tx=0x7fda3000cde0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 shutdown_connections 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fda240a3240 0x7fda240a5700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda3c106f10 0x7fda3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda3c102790 0x7fda3c074a30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 --2- 172.21.15.28:0/3104340021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda3c101de0 0x7fda3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 >> 172.21.15.28:0/3104340021 conn(0x7fda3c0fd8c0 msgr2=0x7fda3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 shutdown_connections 2024-09-19T09:34:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:07.207+0000 7fda41785640 1 -- 172.21.15.28:0/3104340021 wait complete. 2024-09-19T09:34:07.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:07 smithi078 ceph-mon[35619]: pgmap v381: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-19T09:34:07.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:07 smithi038 ceph-mon[35271]: pgmap v381: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-19T09:34:07.849 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:34:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 -- 172.21.15.28:0/3690304864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c074bf0 msgr2=0x7f787c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 --2- 172.21.15.28:0/3690304864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c074bf0 0x7f787c075050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a480 comp rx=0 tx=0).stop 2024-09-19T09:34:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 -- 172.21.15.28:0/3690304864 shutdown_connections 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 --2- 172.21.15.28:0/3690304864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 0x7f787c10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 --2- 172.21.15.28:0/3690304864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c074bf0 0x7f787c075050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 --2- 172.21.15.28:0/3690304864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c0761d0 0x7f787c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 -- 172.21.15.28:0/3690304864 >> 172.21.15.28:0/3690304864 conn(0x7f787c0fd2b0 msgr2=0x7f787c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 -- 172.21.15.28:0/3690304864 shutdown_connections 2024-09-19T09:34:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.062+0000 7f7882e47640 1 -- 172.21.15.28:0/3690304864 wait complete. 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 Processor -- start 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 -- start start 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c074bf0 0x7f787c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 0x7f787c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10f9c0 con 0x7f787c0761d0 2024-09-19T09:34:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10f840 con 0x7f787c1099c0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f7882e47640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10fb40 con 0x7f787c074bf0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f787bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f787bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35950/0 (socket says 172.21.15.28:35950) 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.063+0000 7f787bfff640 1 -- 172.21.15.28:0/3159766694 learned_addr learned my addr 172.21.15.28:0/3159766694 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f78813e4640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 0x7f787c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f7880be3640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c074bf0 0x7f787c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 -- 172.21.15.28:0/3159766694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c074bf0 msgr2=0x7f787c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c074bf0 0x7f787c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 -- 172.21.15.28:0/3159766694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 msgr2=0x7f787c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 0x7f787c1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 -- 172.21.15.28:0/3159766694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c1a1ca0 con 0x7f787c0761d0 2024-09-19T09:34:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f787bfff640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7868009230 tx=0x7f78680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f7879ffb640 1 -- 172.21.15.28:0/3159766694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786803f030 con 0x7f787c0761d0 2024-09-19T09:34:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1a1f30 con 0x7f787c0761d0 2024-09-19T09:34:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f7879ffb640 1 -- 172.21.15.28:0/3159766694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7868003ce0 con 0x7f787c0761d0 2024-09-19T09:34:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.064+0000 7f7879ffb640 1 -- 172.21.15.28:0/3159766694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035070 con 0x7f787c0761d0 2024-09-19T09:34:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.065+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1a23a0 con 0x7f787c0761d0 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.067+0000 7f7879ffb640 1 -- 172.21.15.28:0/3159766694 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7868042030 con 0x7f787c0761d0 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.067+0000 7f7885be0740 1 Processor -- start 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.067+0000 7f7885be0740 1 -- start start 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7879ffb640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 0x7f786c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 0x55a292704da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292707830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a2926632f0 con 0x55a2926ff9a0 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a29260b270 con 0x55a292705370 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7885be0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a29265f2b0 con 0x55a2927002e0 2024-09-19T09:34:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7880be3640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 0x7f786c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35954/0 (socket says 172.21.15.28:35954) 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 -- 172.21.15.28:0/3528694101 learned_addr learned my addr 172.21.15.28:0/3528694101 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7879ffb640 1 -- 172.21.15.28:0/3159766694 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f78680efe40 con 0x7f787c0761d0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f787bfff640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 0x55a292704da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292707830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 -- 172.21.15.28:0/3528694101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 msgr2=0x55a292704da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 0x55a292704da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 -- 172.21.15.28:0/3528694101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 msgr2=0x55a292707830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292707830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 -- 172.21.15.28:0/3528694101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29260a340 con 0x55a2926ff9a0 2024-09-19T09:34:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f787bfff640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 0x55a292704da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f78813e4640 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7874013960 tx=0x7f7874009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d54d08741551bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7880be3640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 0x7f786c0a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7870009b60 tx=0x7f7870005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.068+0000 7f7866ffd640 1 -- 172.21.15.28:0/3528694101 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874004b20 con 0x55a2926ff9a0 2024-09-19T09:34:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7866ffd640 1 -- 172.21.15.28:0/3528694101 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7874003e50 con 0x55a2926ff9a0 2024-09-19T09:34:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7866ffd640 1 -- 172.21.15.28:0/3528694101 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874004250 con 0x55a2926ff9a0 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 -- 172.21.15.28:0/3528694101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 msgr2=0x55a2926ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7874013960 tx=0x7f7874009be0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 -- 172.21.15.28:0/3528694101 shutdown_connections 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292707830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2927002e0 0x55a292704da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 --2- 172.21.15.28:0/3528694101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2926ffda0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.069+0000 7f7885be0740 1 -- 172.21.15.28:0/3528694101 >> 172.21.15.28:0/3528694101 conn(0x55a2926fb010 msgr2=0x55a2926fb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.070+0000 7f7885be0740 1 -- 172.21.15.28:0/3528694101 shutdown_connections 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.070+0000 7f7885be0740 1 -- 172.21.15.28:0/3528694101 wait complete. 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.070+0000 7f7885be0740 1 -- start start 2024-09-19T09:34:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.070+0000 7f7885be0740 1 Processor -- start 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.070+0000 7f7885be0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a292712310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292719890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a2927082e0 con 0x55a2927002e0 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2925ec290 con 0x55a292705370 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a292708460 con 0x55a2926ff9a0 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 10 client.0 _ll_register_callbacks cb 0x55a29261d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35966/0 (socket says 172.21.15.28:35966) 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 -- 172.21.15.28:0/1722624578 learned_addr learned my addr 172.21.15.28:0/1722624578 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f78813e4640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292719890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7880be3640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a292712310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 -- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 msgr2=0x55a292712310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a292712310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 -- 172.21.15.28:0/1722624578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 msgr2=0x55a292719890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292719890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 -- 172.21.15.28:0/1722624578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a292376210 con 0x55a2927002e0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7880be3640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a292712310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f787bfff640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f78680314e0 tx=0x7f78680bc030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba05282869454b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f785bfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868049040 con 0x55a2927002e0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f786803b460 con 0x55a2927002e0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.071+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7874003e50 con 0x55a2927002e0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78680bc430 con 0x55a2927002e0 2024-09-19T09:34:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 10 client.15855 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a292442620 con 0x55a2927002e0 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7874003e50 con 0x55a2927002e0 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78680bb500 con 0x55a2927002e0 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 20 client.15855 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a29271df50 0x55a2927203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 4 client.15855 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.072+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a292670a20 con 0x55a29271df50 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.073+0000 7f7880be3640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a29271df50 0x55a2927203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.073+0000 7f7880be3640 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a29271df50 0x55a2927203a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca47775a8abf2fa3 server_cookie=e748eeefad5d5f1b in_seq=0 out_seq=0 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.073+0000 7f785bfff640 10 client.15855 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f785bfff640 1 -- 172.21.15.28:0/1722624578 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a292670a20 con 0x55a29271df50 2024-09-19T09:34:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f785bfff640 10 client.15855 handle_command_reply: tid=0 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 client.15855 shutdown 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 msgr2=0x55a292712850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f78680314e0 tx=0x7f78680bc030 comp rx=0 tx=0).stop 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 shutdown_connections 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a292705370 0x55a292719890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a29271df50 0x55a2927203a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2927002e0 0x55a292712850 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 --2- 172.21.15.28:0/1722624578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a292712310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 >> 172.21.15.28:0/1722624578 conn(0x55a2926fb010 msgr2=0x55a2926fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f785bfff640 0 client.15855 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 shutdown_connections 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 1 -- 172.21.15.28:0/1722624578 wait complete. 2024-09-19T09:34:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.074+0000 7f7885be0740 20 client.15855 trim_cache size 0 max 16384 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 Processor -- start 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 -- start start 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a2926fbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a292719850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a6d40 con 0x55a292705370 2024-09-19T09:34:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a8880 con 0x55a2927002e0 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7885be0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a7410 con 0x55a2926ff9a0 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35968/0 (socket says 172.21.15.28:35968) 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 -- 172.21.15.28:0/2840606319 learned_addr learned my addr 172.21.15.28:0/2840606319 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f787bfff640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a292719850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f7880be3640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a2926fbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 -- 172.21.15.28:0/2840606319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 msgr2=0x55a2926fbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a2926fbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 -- 172.21.15.28:0/2840606319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 msgr2=0x55a292719850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a292719850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f78813e4640 1 -- 172.21.15.28:0/2840606319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a292720a20 con 0x55a292705370 2024-09-19T09:34:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.094+0000 7f787bfff640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a292719850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.095+0000 7f78813e4640 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7874016010 tx=0x7f78740133f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39faa43204b4c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.095+0000 7f78667fc640 1 -- 172.21.15.28:0/2840606319 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874003c30 con 0x55a292705370 2024-09-19T09:34:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.095+0000 7f78667fc640 1 -- 172.21.15.28:0/2840606319 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7874025030 con 0x55a292705370 2024-09-19T09:34:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.095+0000 7f78667fc640 1 -- 172.21.15.28:0/2840606319 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874020a90 con 0x55a292705370 2024-09-19T09:34:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 -- 172.21.15.28:0/2840606319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 msgr2=0x55a2926fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7874016010 tx=0x7f78740133f0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 -- 172.21.15.28:0/2840606319 shutdown_connections 2024-09-19T09:34:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a292719850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a292705370 0x55a2926fc400 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 --2- 172.21.15.28:0/2840606319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2926ff9a0 0x55a2926fbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 -- 172.21.15.28:0/2840606319 >> 172.21.15.28:0/2840606319 conn(0x55a2926fb010 msgr2=0x55a292701450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 -- 172.21.15.28:0/2840606319 shutdown_connections 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.096+0000 7f7885be0740 1 -- 172.21.15.28:0/2840606319 wait complete. 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 -- start start 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 Processor -- start 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a2927176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292705370 0x55a292718700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a81b0 con 0x55a2926ff9a0 2024-09-19T09:34:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a7ae0 con 0x55a2927002e0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1a4d20 con 0x55a292705370 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7885be0740 10 client.0 _ll_register_callbacks cb 0x7f78680b8750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7880be3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7880be3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35984/0 (socket says 172.21.15.28:35984) 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f7880be3640 1 -- 172.21.15.28:0/3911660203 learned_addr learned my addr 172.21.15.28:0/3911660203 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.097+0000 7f787bfff640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a2927176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f78813e4640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292705370 0x55a292718700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 -- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292705370 msgr2=0x55a292718700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292705370 0x55a292718700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 -- 172.21.15.28:0/3911660203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 msgr2=0x55a2927176f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a2927176f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 -- 172.21.15.28:0/3911660203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a292718d10 con 0x55a2926ff9a0 2024-09-19T09:34:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7880be3640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7870012010 tx=0x7f78700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcda441587f221be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f785bfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870004030 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f78700041d0 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.098+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7874025030 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7870037070 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a292788f20 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7874025030 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78700283b0 con 0x55a2926ff9a0 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 20 client.15861 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a2927898b0 0x55a29278bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 4 client.15861 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a29266ad70 con 0x55a2927898b0 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.099+0000 7f787bfff640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a2927898b0 0x55a29278bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.100+0000 7f785bfff640 10 client.15861 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.100+0000 7f787bfff640 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a2927898b0 0x55a29278bcd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4b9602679e5980 server_cookie=ed44bf76c3dcc144 in_seq=0 out_seq=0 2024-09-19T09:34:08.114 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f785bfff640 1 -- 172.21.15.28:0/3911660203 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a29266ad70 con 0x55a2927898b0 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f785bfff640 10 client.15861 handle_command_reply: tid=0 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 client.15861 shutdown 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 msgr2=0x55a2927171b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7870012010 tx=0x7f78700046c0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 shutdown_connections 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2927002e0 0x55a2927176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a2927898b0 0x55a29278bcd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a292705370 0x55a292718700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f785bfff640 0 client.15861 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 --2- 172.21.15.28:0/3911660203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a2926ff9a0 0x55a2927171b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 >> 172.21.15.28:0/3911660203 conn(0x55a2926fb010 msgr2=0x55a2926f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 shutdown_connections 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 1 -- 172.21.15.28:0/3911660203 wait complete. 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.113+0000 7f7885be0740 20 client.15861 trim_cache size 0 max 16384 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:34:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:34:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 msgr2=0x7f786c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 0x7f786c0a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7870009b60 tx=0x7f7870005f90 comp rx=0 tx=0).stop 2024-09-19T09:34:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 msgr2=0x7f787c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7868009230 tx=0x7f78680042f0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 shutdown_connections 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f786c0a3240 0x7f786c0a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f787c1099c0 0x7f787c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f787c0761d0 0x7f787c19a5c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 --2- 172.21.15.28:0/3159766694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f787c074bf0 0x7f787c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 >> 172.21.15.28:0/3159766694 conn(0x7f787c0fd2b0 msgr2=0x7f787c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 shutdown_connections 2024-09-19T09:34:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:08.114+0000 7f7882e47640 1 -- 172.21.15.28:0/3159766694 wait complete. 2024-09-19T09:34:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 -- 172.21.15.28:0/1387185698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a81036e0 msgr2=0x7f73a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 --2- 172.21.15.28:0/1387185698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a81036e0 0x7f73a8107b90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f739c009930 tx=0x7f739c02a420 comp rx=0 tx=0).stop 2024-09-19T09:34:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 -- 172.21.15.28:0/1387185698 shutdown_connections 2024-09-19T09:34:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 --2- 172.21.15.28:0/1387185698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81082a0 0x7f73a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 --2- 172.21.15.28:0/1387185698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a81036e0 0x7f73a8107b90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 --2- 172.21.15.28:0/1387185698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a8102d30 0x7f73a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.637+0000 7f73af206640 1 -- 172.21.15.28:0/1387185698 >> 172.21.15.28:0/1387185698 conn(0x7f73a80fc7d0 msgr2=0x7f73a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.638+0000 7f73af206640 1 -- 172.21.15.28:0/1387185698 shutdown_connections 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.638+0000 7f73af206640 1 -- 172.21.15.28:0/1387185698 wait complete. 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.638+0000 7f73af206640 1 Processor -- start 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.638+0000 7f73af206640 1 -- start start 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81036e0 0x7f73a8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a81082a0 0x7f73a819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81120b0 con 0x7f73a8102d30 2024-09-19T09:34:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8111f30 con 0x7f73a81036e0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73af206640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8112230 con 0x7f73a81082a0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36002/0 (socket says 172.21.15.28:36002) 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 -- 172.21.15.28:0/3657502995 learned_addr learned my addr 172.21.15.28:0/3657502995 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73a7fff640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81036e0 0x7f73a8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a81082a0 0x7f73a819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 -- 172.21.15.28:0/3657502995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a81082a0 msgr2=0x7f73a819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a81082a0 0x7f73a819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 -- 172.21.15.28:0/3657502995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81036e0 msgr2=0x7f73a8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81036e0 0x7f73a8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 -- 172.21.15.28:0/3657502995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a819d560 con 0x7f73a8102d30 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.639+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f739800c970 tx=0x7f739800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.640+0000 7f73a5ffb640 1 -- 172.21.15.28:0/3657502995 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398007bb0 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.640+0000 7f73a5ffb640 1 -- 172.21.15.28:0/3657502995 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7398007d50 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.640+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a819d850 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.640+0000 7f73a5ffb640 1 -- 172.21.15.28:0/3657502995 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739800f450 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.640+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a5130 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.642+0000 7f73a5ffb640 1 -- 172.21.15.28:0/3657502995 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7398005020 con 0x7f73a8102d30 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 Processor -- start 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 -- start start 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a5ffb640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5624a2cf02f0 con 0x5624a2d8a5d0 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624a2c97cf0 con 0x5624a2d8bb40 2024-09-19T09:34:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73b15df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624a2cec2b0 con 0x5624a2d95370 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a5ffb640 1 -- 172.21.15.28:0/3657502995 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73980cf670 con 0x7f73a8102d30 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36018/0 (socket says 172.21.15.28:36018) 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 -- 172.21.15.28:0/63344678 learned_addr learned my addr 172.21.15.28:0/63344678 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73acfa2640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 -- 172.21.15.28:0/63344678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 msgr2=0x5624a2d97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d97760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 -- 172.21.15.28:0/63344678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 msgr2=0x5624a2d8a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d8a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73a7fff640 1 -- 172.21.15.28:0/63344678 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a2c7c3a0 con 0x5624a2d8a5d0 2024-09-19T09:34:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 0x7f73900a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.643+0000 7f73acfa2640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f73a7fff640 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989e1a3f85247cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f738effd640 1 -- 172.21.15.28:0/63344678 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c040070 con 0x5624a2d8a5d0 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f738effd640 1 -- 172.21.15.28:0/63344678 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739c005500 con 0x5624a2d8a5d0 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f738effd640 1 -- 172.21.15.28:0/63344678 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c035040 con 0x5624a2d8a5d0 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f73b15df740 1 -- 172.21.15.28:0/63344678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 msgr2=0x5624a2d8aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.644+0000 7f73b15df740 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c004170 comp rx=0 tx=0).stop 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- 172.21.15.28:0/63344678 shutdown_connections 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d8aa10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- 172.21.15.28:0/63344678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- 172.21.15.28:0/63344678 >> 172.21.15.28:0/63344678 conn(0x5624a2d53310 msgr2=0x5624a2d86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- 172.21.15.28:0/63344678 shutdown_connections 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- 172.21.15.28:0/63344678 wait complete. 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- start start 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 Processor -- start 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5624a2d98010 con 0x5624a2d8a5d0 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624a2cab060 con 0x5624a2d8bb40 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.645+0000 7f73b15df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624a2d98190 con 0x5624a2d95370 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73b15df740 10 client.0 _ll_register_callbacks cb 0x5624a29fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36024/0 (socket says 172.21.15.28:36024) 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 -- 172.21.15.28:0/110604046 learned_addr learned my addr 172.21.15.28:0/110604046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73a7fff640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 -- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 msgr2=0x5624a2d927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 -- 172.21.15.28:0/110604046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 msgr2=0x5624a2d92050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d92050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 -- 172.21.15.28:0/110604046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a2a613c0 con 0x5624a2d8a5d0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73acfa2640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f73980986e0 tx=0x7f739807f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea358a4d1966c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f7387fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739801d040 con 0x5624a2d8a5d0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f73980cd250 con 0x5624a2d8a5d0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c005500 con 0x5624a2d8a5d0 2024-09-19T09:34:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f739807f9e0 con 0x5624a2d8a5d0 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73b15df740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.646+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624a2da3b50 con 0x5624a2d8a5d0 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73980b1080 con 0x5624a2d8a5d0 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f739809b160 con 0x5624a2d8a5d0 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 20 client.15873 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2da4f60 0x5624a2da7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 4 client.15873 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624a2cfd860 con 0x5624a2da4f60 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.647+0000 7f73a7fff640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2da4f60 0x5624a2da7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.648+0000 7f73a7fff640 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2da4f60 0x5624a2da7350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66897e417f5bb4db server_cookie=60515872f7c7d3a3 in_seq=0 out_seq=0 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.648+0000 7f7387fff640 10 client.15873 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.648+0000 7f7387fff640 1 -- 172.21.15.28:0/110604046 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624a2cfd860 con 0x5624a2da4f60 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.648+0000 7f7387fff640 10 client.15873 handle_command_reply: tid=0 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 client.15873 shutdown 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 msgr2=0x5624a2d94da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f73980986e0 tx=0x7f739807f8c0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 shutdown_connections 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2da4f60 0x5624a2da7350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f7387fff640 0 client.15873 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 --2- 172.21.15.28:0/110604046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d94da0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 >> 172.21.15.28:0/110604046 conn(0x5624a2d53310 msgr2=0x5624a2d8c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 shutdown_connections 2024-09-19T09:34:08.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 1 -- 172.21.15.28:0/110604046 wait complete. 2024-09-19T09:34:08.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.649+0000 7f73b15df740 20 client.15873 trim_cache size 0 max 16384 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 Processor -- start 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 -- start start 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8a5d0 0x5624a2d85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d8bb40 0x5624a2d8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ad4c0 con 0x5624a2d95370 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81af000 con 0x5624a2d8a5d0 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73b15df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81adb90 con 0x5624a2d8bb40 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36040/0 (socket says 172.21.15.28:36040) 2024-09-19T09:34:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 -- 172.21.15.28:0/125196056 learned_addr learned my addr 172.21.15.28:0/125196056 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73acfa2640 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8a5d0 0x5624a2d85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73a7fff640 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d8bb40 0x5624a2d8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 -- 172.21.15.28:0/125196056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d8bb40 msgr2=0x5624a2d8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d8bb40 0x5624a2d8d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 -- 172.21.15.28:0/125196056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8a5d0 msgr2=0x5624a2d85c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8a5d0 0x5624a2d85c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.663+0000 7f73ad7a3640 1 -- 172.21.15.28:0/125196056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a2d905a0 con 0x5624a2d95370 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f73a0021810 tx=0x7f73a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483fe0551243a847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f738e7fc640 1 -- 172.21.15.28:0/125196056 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0013b80 con 0x5624a2d95370 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f738e7fc640 1 -- 172.21.15.28:0/125196056 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a0013d20 con 0x5624a2d95370 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f738e7fc640 1 -- 172.21.15.28:0/125196056 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a001d810 con 0x5624a2d95370 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 -- 172.21.15.28:0/125196056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 msgr2=0x5624a2d8d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f73a0021810 tx=0x7f73a0015000 comp rx=0 tx=0).stop 2024-09-19T09:34:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 -- 172.21.15.28:0/125196056 shutdown_connections 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d95370 0x5624a2d8d660 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d8bb40 0x5624a2d8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 --2- 172.21.15.28:0/125196056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8a5d0 0x5624a2d85c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.664+0000 7f73b15df740 1 -- 172.21.15.28:0/125196056 >> 172.21.15.28:0/125196056 conn(0x5624a2d53310 msgr2=0x5624a2d85500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- 172.21.15.28:0/125196056 shutdown_connections 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- 172.21.15.28:0/125196056 wait complete. 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- start start 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 Processor -- start 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d99920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d99e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae930 con 0x5624a2d8a5d0 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae260 con 0x5624a2d8bb40 2024-09-19T09:34:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ab4a0 con 0x5624a2d95370 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73b15df740 10 client.0 _ll_register_callbacks cb 0x7f739809b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.665+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36042/0 (socket says 172.21.15.28:36042) 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 -- 172.21.15.28:0/3645988056 learned_addr learned my addr 172.21.15.28:0/3645988056 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d99e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73a7fff640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d99920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 -- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 msgr2=0x5624a2d99e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d99e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 -- 172.21.15.28:0/3645988056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 msgr2=0x5624a2d99920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d99920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 -- 172.21.15.28:0/3645988056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a2e0ee90 con 0x5624a2d8a5d0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73ad7a3640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d99e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73acfa2640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f739809bca0 tx=0x7f7398005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434f6619163399de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f7387fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739801d070 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f739809b160 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a0013d20 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73980b9050 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73b15df740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.666+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624a2e10190 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a0013d20 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7398097af0 con 0x5624a2d8a5d0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 20 client.15879 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2e108b0 0x5624a2e12ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 4 client.15879 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624a2cf7d70 con 0x5624a2e108b0 2024-09-19T09:34:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.667+0000 7f73a7fff640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2e108b0 0x5624a2e12ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73a7fff640 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2e108b0 0x5624a2e12ca0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b77f95fd54a9394 server_cookie=f0cccfdae994ef71 in_seq=0 out_seq=0 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f7387fff640 10 client.15879 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f7387fff640 1 -- 172.21.15.28:0/3645988056 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624a2cf7d70 con 0x5624a2e108b0 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f7387fff640 10 client.15879 handle_command_reply: tid=0 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 client.15879 shutdown 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 msgr2=0x5624a2d993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f739809bca0 tx=0x7f7398005700 comp rx=0 tx=0).stop 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 shutdown_connections 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x5624a2e108b0 0x5624a2e12ca0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624a2d8bb40 0x5624a2d99920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624a2d95370 0x5624a2d99e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 --2- 172.21.15.28:0/3645988056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624a2d8a5d0 0x5624a2d993e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f7387fff640 0 client.15879 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:08.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 >> 172.21.15.28:0/3645988056 conn(0x5624a2d53310 msgr2=0x5624a2d940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 shutdown_connections 2024-09-19T09:34:08.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 1 -- 172.21.15.28:0/3645988056 wait complete. 2024-09-19T09:34:08.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.668+0000 7f73b15df740 20 client.15879 trim_cache size 0 max 16384 2024-09-19T09:34:08.677 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:08.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 msgr2=0x7f73900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 0x7f73900a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009a60 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 msgr2=0x7f73a81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f739800c970 tx=0x7f739800ce30 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 shutdown_connections 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73a81036e0 0x7f73a8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a81082a0 0x7f73a819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 --2- 172.21.15.28:0/3657502995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73a8102d30 0x7f73a81985b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 >> 172.21.15.28:0/3657502995 conn(0x7f73a80fc7d0 msgr2=0x7f73a8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 shutdown_connections 2024-09-19T09:34:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:08.669+0000 7f73af206640 1 -- 172.21.15.28:0/3657502995 wait complete. 2024-09-19T09:34:09.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:09 smithi028 ceph-mon[30972]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 70 KiB/s wr, 27 op/s 2024-09-19T09:34:09.355 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:34:09.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:09 smithi078 ceph-mon[35619]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 70 KiB/s wr, 27 op/s 2024-09-19T09:34:09.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:09 smithi038 ceph-mon[35271]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 70 KiB/s wr, 27 op/s 2024-09-19T09:34:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 -- 172.21.15.28:0/3409655275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294108290 msgr2=0x7ff29410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 --2- 172.21.15.28:0/3409655275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294108290 0x7ff29410ec30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009930 tx=0x7ff28c02a420 comp rx=0 tx=0).stop 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 -- 172.21.15.28:0/3409655275 shutdown_connections 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 --2- 172.21.15.28:0/3409655275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294103740 0x7ff294107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 --2- 172.21.15.28:0/3409655275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294108290 0x7ff29410ec30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 --2- 172.21.15.28:0/3409655275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 0x7ff294103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 -- 172.21.15.28:0/3409655275 >> 172.21.15.28:0/3409655275 conn(0x7ff2940fc7d0 msgr2=0x7ff2940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 -- 172.21.15.28:0/3409655275 shutdown_connections 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.064+0000 7ff298e31640 1 -- 172.21.15.28:0/3409655275 wait complete. 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 Processor -- start 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 -- start start 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 0x7ff294198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294108290 0x7ff29419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff294112090 con 0x7ff294103740 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff294111f10 con 0x7ff294108290 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff298e31640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff294112210 con 0x7ff294102d90 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff291d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff291d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36054/0 (socket says 172.21.15.28:36054) 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff291d9b640 1 -- 172.21.15.28:0/754051017 learned_addr learned my addr 172.21.15.28:0/754051017 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff292d9d640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294108290 0x7ff29419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.065+0000 7ff29259c640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 0x7ff294198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 -- 172.21.15.28:0/754051017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 msgr2=0x7ff294198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 0x7ff294198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 -- 172.21.15.28:0/754051017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294108290 msgr2=0x7ff29419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294108290 0x7ff29419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 -- 172.21.15.28:0/754051017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29419d440 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff291d9b640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c970 tx=0x7ff28800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff2837fe640 1 -- 172.21.15.28:0/754051017 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288014070 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29419d730 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff2837fe640 1 -- 172.21.15.28:0/754051017 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff288004440 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff2837fe640 1 -- 172.21.15.28:0/754051017 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28800f450 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.066+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2941a5010 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.068+0000 7ff2837fe640 1 -- 172.21.15.28:0/754051017 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff288017070 con 0x7ff294103740 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 Processor -- start 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 -- start start 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3ffeba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563c3ff512f0 con 0x563c3ffecb40 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c3fef8cf0 con 0x563c3ffeb5d0 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff29b20a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563c3ff4d2b0 con 0x563c3fff6370 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36056/0 (socket says 172.21.15.28:36056) 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.069+0000 7ff292d9d640 1 -- 172.21.15.28:0/748512448 learned_addr learned my addr 172.21.15.28:0/748512448 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff291d9b640 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff2837fe640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 0x7ff27c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff29259c640 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3ffeba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 -- 172.21.15.28:0/748512448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 msgr2=0x563c3fff8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 -- 172.21.15.28:0/748512448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 msgr2=0x563c3ffeba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3ffeba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 -- 172.21.15.28:0/748512448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3fedd3a0 con 0x563c3ffecb40 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff2837fe640 1 -- 172.21.15.28:0/754051017 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff2880cf680 con 0x7ff294103740 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 0x7ff27c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff292d9d640 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009930 tx=0x7ff28c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ee334a311d7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff280ff9640 1 -- 172.21.15.28:0/748512448 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c035040 con 0x563c3ffecb40 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff280ff9640 1 -- 172.21.15.28:0/748512448 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c004380 con 0x563c3ffecb40 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.070+0000 7ff280ff9640 1 -- 172.21.15.28:0/748512448 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c046ce0 con 0x563c3ffecb40 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 -- 172.21.15.28:0/748512448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 msgr2=0x563c3ffeb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009930 tx=0x7ff28c0317e0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff292d9d640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 0x7ff27c0a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff28c04b000 tx=0x7ff28c048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 -- 172.21.15.28:0/748512448 shutdown_connections 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3ffeba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 --2- 172.21.15.28:0/748512448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3ffeb090 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 -- 172.21.15.28:0/748512448 >> 172.21.15.28:0/748512448 conn(0x563c3ffb4310 msgr2=0x563c3ffe7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 -- 172.21.15.28:0/748512448 shutdown_connections 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.071+0000 7ff29b20a740 1 -- 172.21.15.28:0/748512448 wait complete. 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 -- start start 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 Processor -- start 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c3fff5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3fff3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563c3fff9070 con 0x563c3fff6370 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563c3ff0c060 con 0x563c3ffecb40 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563c3fff91f0 con 0x563c3ffeb5d0 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29b20a740 10 client.0 _ll_register_callbacks cb 0x563c3fc5f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36066/0 (socket says 172.21.15.28:36066) 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff292d9d640 1 -- 172.21.15.28:0/4041520921 learned_addr learned my addr 172.21.15.28:0/4041520921 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff291d9b640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3fff3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.072+0000 7ff29259c640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c3fff5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 -- 172.21.15.28:0/4041520921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 msgr2=0x563c3fff5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c3fff5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 -- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 msgr2=0x563c3fff3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3fff3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 -- 172.21.15.28:0/4041520921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c3fcc23c0 con 0x563c3fff6370 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff292d9d640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006010 tx=0x7ff28c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddccff9a04112484 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c059020 con 0x563c3fff6370 2024-09-19T09:34:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff28c046cb0 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c004380 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff28c003a40 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff29b20a740 10 client.15891 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c3fff9070 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.073+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c004380 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff28c051c60 con 0x563c3fff6370 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 20 client.15891 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40005c90 0x563c40008080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 4 client.15891 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c3ff5e860 con 0x563c40005c90 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.074+0000 7ff29259c640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40005c90 0x563c40008080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.075+0000 7ff29259c640 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40005c90 0x563c40008080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab2bcb2de3a40211 server_cookie=a3098fe40567f855 in_seq=0 out_seq=0 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.075+0000 7ff279ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.075+0000 7ff279ffb640 1 -- 172.21.15.28:0/4041520921 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c3ff5e860 con 0x563c40005c90 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.075+0000 7ff279ffb640 10 client.15891 handle_command_reply: tid=0 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.075+0000 7ff29b20a740 1 client.15891 shutdown 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 msgr2=0x563c3fff3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006010 tx=0x7ff28c009290 comp rx=0 tx=0).stop 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 shutdown_connections 2024-09-19T09:34:10.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40005c90 0x563c40008080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3fff3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3fff3610 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff279ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 --2- 172.21.15.28:0/4041520921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c3fff5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 >> 172.21.15.28:0/4041520921 conn(0x563c3ffb4310 msgr2=0x563c3ffed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 shutdown_connections 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 1 -- 172.21.15.28:0/4041520921 wait complete. 2024-09-19T09:34:10.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.076+0000 7ff29b20a740 20 client.15891 trim_cache size 0 max 16384 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 Processor -- start 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 -- start start 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c40002310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3ffee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941ad480 con 0x563c3fff6370 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941aefc0 con 0x563c3ffecb40 2024-09-19T09:34:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff29b20a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941adb50 con 0x563c3ffeb5d0 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.089+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36072/0 (socket says 172.21.15.28:36072) 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 -- 172.21.15.28:0/634120031 learned_addr learned my addr 172.21.15.28:0/634120031 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff291d9b640 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3ffee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff29259c640 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c40002310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 -- 172.21.15.28:0/634120031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 msgr2=0x563c40002310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c40002310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 -- 172.21.15.28:0/634120031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 msgr2=0x563c3ffee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3ffee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 -- 172.21.15.28:0/634120031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3ffe8960 con 0x563c3fff6370 2024-09-19T09:34:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff292d9d640 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009600 tx=0x7ff28c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02fd48e679b1e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff27bfff640 1 -- 172.21.15.28:0/634120031 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c035030 con 0x563c3fff6370 2024-09-19T09:34:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff27bfff640 1 -- 172.21.15.28:0/634120031 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c004380 con 0x563c3fff6370 2024-09-19T09:34:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.090+0000 7ff27bfff640 1 -- 172.21.15.28:0/634120031 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c059070 con 0x563c3fff6370 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- 172.21.15.28:0/634120031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 msgr2=0x563c3ffee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009600 tx=0x7ff28c009300 comp rx=0 tx=0).stop 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- 172.21.15.28:0/634120031 shutdown_connections 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffecb40 0x563c3ffee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3fff6370 0x563c3ffee660 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 --2- 172.21.15.28:0/634120031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ffeb5d0 0x563c40002310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- 172.21.15.28:0/634120031 >> 172.21.15.28:0/634120031 conn(0x563c3ffb4310 msgr2=0x563c3fff2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- 172.21.15.28:0/634120031 shutdown_connections 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- 172.21.15.28:0/634120031 wait complete. 2024-09-19T09:34:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 -- start start 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.091+0000 7ff29b20a740 1 Processor -- start 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3fff00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941ae8f0 con 0x563c3ffecb40 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941ae220 con 0x563c3ffeb5d0 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941ab460 con 0x563c3fff6370 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29b20a740 10 client.0 _ll_register_callbacks cb 0x7ff28c03f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36084/0 (socket says 172.21.15.28:36084) 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 -- 172.21.15.28:0/2147482844 learned_addr learned my addr 172.21.15.28:0/2147482844 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29259c640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3fff00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff292d9d640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 -- 172.21.15.28:0/2147482844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 msgr2=0x563c3fff0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 -- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 msgr2=0x563c3fff00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3fff00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 -- 172.21.15.28:0/2147482844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4006f7b0 con 0x563c3ffecb40 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff29259c640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3fff00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff292d9d640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff291d9b640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff288007590 tx=0x7ff28809bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437b8b05b9db4be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.092+0000 7ff279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880a48e0 con 0x563c3ffecb40 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff2880d1b40 con 0x563c3ffecb40 2024-09-19T09:34:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c004380 con 0x563c3ffecb40 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2880b9050 con 0x563c3ffecb40 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c40070af0 con 0x563c3ffecb40 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c004380 con 0x563c3ffecb40 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff2880860d0 con 0x563c3ffecb40 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 20 client.15897 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40071220 0x563c40073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 4 client.15897 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c3ff58d70 con 0x563c40071220 2024-09-19T09:34:10.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.093+0000 7ff29259c640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40071220 0x563c40073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff279ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29259c640 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40071220 0x563c40073640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86b94d85c43e0ed3 server_cookie=39d3fb6a30eeeae7 in_seq=0 out_seq=0 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff279ffb640 1 -- 172.21.15.28:0/2147482844 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c3ff58d70 con 0x563c40071220 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff279ffb640 10 client.15897 handle_command_reply: tid=0 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 client.15897 shutdown 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 msgr2=0x563c3fff0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff288007590 tx=0x7ff28809bd30 comp rx=0 tx=0).stop 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 shutdown_connections 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3fff6370 0x563c3fff0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563c3ffecb40 0x563c3fff0630 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x563c40071220 0x563c40073640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 --2- 172.21.15.28:0/2147482844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563c3ffeb5d0 0x563c3fff00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 >> 172.21.15.28:0/2147482844 conn(0x563c3ffb4310 msgr2=0x563c3fff4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff279ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:10.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 shutdown_connections 2024-09-19T09:34:10.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 1 -- 172.21.15.28:0/2147482844 wait complete. 2024-09-19T09:34:10.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.094+0000 7ff29b20a740 20 client.15897 trim_cache size 0 max 16384 2024-09-19T09:34:10.103 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 msgr2=0x7ff27c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 0x7ff27c0a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff28c04b000 tx=0x7ff28c048000 comp rx=0 tx=0).stop 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 msgr2=0x7ff2941989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c970 tx=0x7ff28800ce30 comp rx=0 tx=0).stop 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 shutdown_connections 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff27c0a3240 0x7ff27c0a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff294108290 0x7ff29419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff294103740 0x7ff2941989d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 --2- 172.21.15.28:0/754051017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff294102d90 0x7ff294198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:10.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 >> 172.21.15.28:0/754051017 conn(0x7ff2940fc7d0 msgr2=0x7ff2940fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:10.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 shutdown_connections 2024-09-19T09:34:10.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:10.095+0000 7ff298e31640 1 -- 172.21.15.28:0/754051017 wait complete. 2024-09-19T09:34:10.397 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:34:10.772 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:34:10.838 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:11.315 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:11 smithi028 ceph-mon[30972]: pgmap v383: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 -- 172.21.15.28:0/1735023153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc1009b0 msgr2=0x7f04cc100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 --2- 172.21.15.28:0/1735023153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc1009b0 0x7f04cc100d90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009900 tx=0x7f04bc02a410 comp rx=0 tx=0).stop 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 -- 172.21.15.28:0/1735023153 shutdown_connections 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 --2- 172.21.15.28:0/1735023153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc101360 0x7f04cc10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 --2- 172.21.15.28:0/1735023153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc10e4d0 0x7f04cc1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 --2- 172.21.15.28:0/1735023153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc1009b0 0x7f04cc100d90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 -- 172.21.15.28:0/1735023153 >> 172.21.15.28:0/1735023153 conn(0x7f04cc0fc7d0 msgr2=0x7f04cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 -- 172.21.15.28:0/1735023153 shutdown_connections 2024-09-19T09:34:11.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.321+0000 7f04d2718640 1 -- 172.21.15.28:0/1735023153 wait complete. 2024-09-19T09:34:11.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 Processor -- start 2024-09-19T09:34:11.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 -- start start 2024-09-19T09:34:11.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc1009b0 0x7f04cc19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:11.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 0x7f04cc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:11.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc114260 con 0x7f04cc10e4d0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1140e0 con 0x7f04cc1009b0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d2718640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1143e0 con 0x7f04cc101360 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d0cb5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d0cb5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36098/0 (socket says 172.21.15.28:36098) 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.322+0000 7f04d0cb5640 1 -- 172.21.15.28:0/1875521984 learned_addr learned my addr 172.21.15.28:0/1875521984 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04cbfff640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc1009b0 0x7f04cc19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04cb7fe640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 0x7f04cc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 -- 172.21.15.28:0/1875521984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 msgr2=0x7f04cc19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 0x7f04cc19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 -- 172.21.15.28:0/1875521984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc1009b0 msgr2=0x7f04cc19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc1009b0 0x7f04cc19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 -- 172.21.15.28:0/1875521984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc19f6f0 con 0x7f04cc10e4d0 2024-09-19T09:34:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04cb7fe640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 0x7f04cc19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d0cb5640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f04c400eb10 tx=0x7f04c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004b20 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c4004cc0 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc19f9e0 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.323+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4010760 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.324+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc1a72c0 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.325+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc06a7f0 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.326+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f04c4005020 con 0x7f04cc10e4d0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.327+0000 7f04c97fa640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 0x7f04b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.327+0000 7f04cbfff640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 0x7f04b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.327+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f04c40cfe30 con 0x7f04cc10e4d0 2024-09-19T09:34:11.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.328+0000 7f04cbfff640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 0x7f04b40a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:11.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.331+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04c409cb20 con 0x7f04cc10e4d0 2024-09-19T09:34:11.355 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:11.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:11 smithi078 ceph-mon[35619]: pgmap v383: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-09-19T09:34:11.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:11 smithi038 ceph-mon[35271]: pgmap v383: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-09-19T09:34:11.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.550+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04cc109660 con 0x7f04cc10e4d0 2024-09-19T09:34:11.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.551+0000 7f04c97fa640 1 -- 172.21.15.28:0/1875521984 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7f04c409c420 con 0x7f04cc10e4d0 2024-09-19T09:34:11.553 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:34:11.553 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:34:11.553 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:34:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 msgr2=0x7f04b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 0x7f04b40a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005b50 comp rx=0 tx=0).stop 2024-09-19T09:34:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 msgr2=0x7f04cc19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f04c400eb10 tx=0x7f04c400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 shutdown_connections 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04b40a3240 0x7f04b40a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04cc10e4d0 0x7f04cc19f010 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04cc101360 0x7f04cc19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 --2- 172.21.15.28:0/1875521984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04cc1009b0 0x7f04cc19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 >> 172.21.15.28:0/1875521984 conn(0x7f04cc0fc7d0 msgr2=0x7f04cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 shutdown_connections 2024-09-19T09:34:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:11.553+0000 7f04d2718640 1 -- 172.21.15.28:0/1875521984 wait complete. 2024-09-19T09:34:12.195 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id 5bd3b1f --release --await' 2024-09-19T09:34:12.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id 5bd3b1f --release --await 2024-09-19T09:34:12.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:12 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1875521984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:12.366 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:12 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1875521984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:12.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 -- 172.21.15.28:0/3105089424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 msgr2=0x7f1c6c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 --2- 172.21.15.28:0/3105089424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c100ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009900 tx=0x7f1c6002a410 comp rx=0 tx=0).stop 2024-09-19T09:34:12.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 -- 172.21.15.28:0/3105089424 shutdown_connections 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 --2- 172.21.15.28:0/3105089424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 --2- 172.21.15.28:0/3105089424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 --2- 172.21.15.28:0/3105089424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c100ca0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 -- 172.21.15.28:0/3105089424 >> 172.21.15.28:0/3105089424 conn(0x7f1c6c0fc7d0 msgr2=0x7f1c6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 -- 172.21.15.28:0/3105089424 shutdown_connections 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.365+0000 7f1c73798640 1 -- 172.21.15.28:0/3105089424 wait complete. 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 Processor -- start 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 -- start start 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114180 con 0x7f1c6c1008c0 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114000 con 0x7f1c6c101270 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c73798640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114300 con 0x7f1c6c10e470 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36120/0 (socket says 172.21.15.28:36120) 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c71534640 1 -- 172.21.15.28:0/1897264128 learned_addr learned my addr 172.21.15.28:0/1897264128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.366+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 -- 172.21.15.28:0/1897264128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 msgr2=0x7f1c6c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 -- 172.21.15.28:0/1897264128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 msgr2=0x7f1c6c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 -- 172.21.15.28:0/1897264128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19f650 con 0x7f1c6c1008c0 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c71534640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1c60005ab0 tx=0x7f1c600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c5e7fc640 1 -- 172.21.15.28:0/1897264128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6003f030 con 0x7f1c6c1008c0 2024-09-19T09:34:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c5e7fc640 1 -- 172.21.15.28:0/1897264128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c60003ce0 con 0x7f1c6c1008c0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19f8e0 con 0x7f1c6c1008c0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c5e7fc640 1 -- 172.21.15.28:0/1897264128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60035070 con 0x7f1c6c1008c0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.367+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a7220 con 0x7f1c6c1008c0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.370+0000 7f1c5e7fc640 1 -- 172.21.15.28:0/1897264128 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1c60042030 con 0x7f1c6c1008c0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.370+0000 7f1c75b71740 1 Processor -- start 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.370+0000 7f1c75b71740 1 -- start start 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.370+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302550090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.370+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x56130255d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c75b71740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5613024b62f0 con 0x5613025505d0 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c75b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56130245dcf0 con 0x56130255b370 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c75b71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5613024b22b0 con 0x561302551b40 2024-09-19T09:34:12.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c5e7fc640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 0x7f1c580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36124/0 (socket says 172.21.15.28:36124) 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 -- 172.21.15.28:0/1694688293 learned_addr learned my addr 172.21.15.28:0/1694688293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302550090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x56130255d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 0x7f1c580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c5e7fc640 1 -- 172.21.15.28:0/1897264128 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1c600f0220 con 0x7f1c6c1008c0 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 -- 172.21.15.28:0/1694688293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 msgr2=0x561302550090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302550090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 -- 172.21.15.28:0/1694688293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 msgr2=0x56130255d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x56130255d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 -- 172.21.15.28:0/1694688293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613024423a0 con 0x5613025505d0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.371+0000 7f1c71534640 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c600edd50 tx=0x7f1c6002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53da78c388f44f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.372+0000 7f1c577fe640 1 -- 172.21.15.28:0/1694688293 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60049030 con 0x5613025505d0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.372+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 0x7f1c580a5780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400afd0 tx=0x7f1c64005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.372+0000 7f1c577fe640 1 -- 172.21.15.28:0/1694688293 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c600a7070 con 0x5613025505d0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.372+0000 7f1c577fe640 1 -- 172.21.15.28:0/1694688293 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c600bac00 con 0x5613025505d0 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- 172.21.15.28:0/1694688293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 msgr2=0x561302550a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c600edd50 tx=0x7f1c6002a410 comp rx=0 tx=0).stop 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- 172.21.15.28:0/1694688293 shutdown_connections 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x56130255d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302550a10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1694688293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302550090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- 172.21.15.28:0/1694688293 >> 172.21.15.28:0/1694688293 conn(0x561302519310 msgr2=0x56130254cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- 172.21.15.28:0/1694688293 shutdown_connections 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- 172.21.15.28:0/1694688293 wait complete. 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 -- start start 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.373+0000 7f1c75b71740 1 Processor -- start 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x561302558600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5613024b62f0 con 0x561302551b40 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561302471060 con 0x56130255b370 2024-09-19T09:34:12.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56130255de30 con 0x5613025505d0 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36138/0 (socket says 172.21.15.28:36138) 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 -- 172.21.15.28:0/1296194418 learned_addr learned my addr 172.21.15.28:0/1296194418 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c75b71740 10 client.0 _ll_register_callbacks cb 0x5613021c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c71534640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x561302558600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 -- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 msgr2=0x56130255ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 -- 172.21.15.28:0/1296194418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 msgr2=0x561302558600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x561302558600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c70d33640 1 -- 172.21.15.28:0/1296194418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613022273c0 con 0x561302551b40 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.374+0000 7f1c71d35640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x561302558600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:12.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c70d33640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c68011800 tx=0x7f1c68011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5951e086b7915e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800f9a0 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c600a7070 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1c6800fb40 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c6800fd80 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c75b71740 10 client.15915 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561302571ff0 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c600a7070 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.375+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c68018030 con 0x561302551b40 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 10 client.15915 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 20 client.15915 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025734a0 0x5613025758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 4 client.15915 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613024c3860 con 0x5613025734a0 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c71534640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025734a0 0x5613025758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c71534640 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025734a0 0x5613025758b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83d147a63265b5 server_cookie=a4d9a86588195420 in_seq=0 out_seq=0 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.376+0000 7f1c54ff9640 10 client.15915 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c54ff9640 1 -- 172.21.15.28:0/1296194418 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613024c3860 con 0x5613025734a0 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c54ff9640 10 client.15915 handle_command_reply: tid=0 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 client.15915 shutdown 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 msgr2=0x561302558010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c68011800 tx=0x7f1c68011cc0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 shutdown_connections 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x561302558600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.377+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025734a0 0x5613025758b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c54ff9640 0 client.15915 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561302551b40 0x561302558010 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 1 --2- 172.21.15.28:0/1296194418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 >> 172.21.15.28:0/1296194418 conn(0x561302519310 msgr2=0x561302552990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 shutdown_connections 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 1 -- 172.21.15.28:0/1296194418 wait complete. 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.378+0000 7f1c75b71740 20 client.15915 trim_cache size 0 max 16384 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 Processor -- start 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 -- start start 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302553180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x5613025536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ab380 con 0x5613025505d0 2024-09-19T09:34:12.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1acec0 con 0x56130255b370 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c75b71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1aba50 con 0x561302551b40 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c71534640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36140/0 (socket says 172.21.15.28:36140) 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.392+0000 7f1c71534640 1 -- 172.21.15.28:0/787041809 learned_addr learned my addr 172.21.15.28:0/787041809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c70d33640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302553180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71d35640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x5613025536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 -- 172.21.15.28:0/787041809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 msgr2=0x561302553180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302553180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 -- 172.21.15.28:0/787041809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 msgr2=0x5613025536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x5613025536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 -- 172.21.15.28:0/787041809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561302553c00 con 0x5613025505d0 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71d35640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x5613025536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c71534640 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006010 tx=0x7f1c60046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b867b6fd60a1681f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c56ffd640 1 -- 172.21.15.28:0/787041809 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60049040 con 0x5613025505d0 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c56ffd640 1 -- 172.21.15.28:0/787041809 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c600bcf70 con 0x5613025505d0 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.393+0000 7f1c56ffd640 1 -- 172.21.15.28:0/787041809 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6003b460 con 0x5613025505d0 2024-09-19T09:34:12.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- 172.21.15.28:0/787041809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 msgr2=0x561302566f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006010 tx=0x7f1c60046040 comp rx=0 tx=0).stop 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- 172.21.15.28:0/787041809 shutdown_connections 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56130255b370 0x5613025536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561302551b40 0x561302553180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 --2- 172.21.15.28:0/787041809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5613025505d0 0x561302566f50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- 172.21.15.28:0/787041809 >> 172.21.15.28:0/787041809 conn(0x561302519310 msgr2=0x561302557100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- 172.21.15.28:0/787041809 shutdown_connections 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- 172.21.15.28:0/787041809 wait complete. 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 -- start start 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.394+0000 7f1c75b71740 1 Processor -- start 2024-09-19T09:34:12.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561302551b40 0x56130255f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ac7f0 con 0x56130255b370 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ac120 con 0x561302551b40 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1a9360 con 0x5613025505d0 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c75b71740 10 client.0 _ll_register_callbacks cb 0x7f1c68008c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36150/0 (socket says 172.21.15.28:36150) 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 -- 172.21.15.28:0/2393056105 learned_addr learned my addr 172.21.15.28:0/2393056105 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c70d33640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561302551b40 0x56130255f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71534640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 -- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 msgr2=0x56130255edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 -- 172.21.15.28:0/2393056105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561302551b40 msgr2=0x56130255f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561302551b40 0x56130255f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71d35640 1 -- 172.21.15.28:0/2393056105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613025dd190 con 0x56130255b370 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.395+0000 7f1c71534640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c71d35640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1c64010b40 tx=0x7f1c64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669aa9bb4d91728b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:12.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64004b20 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1c64004cc0 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c600bcf70 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c640127d0 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 10 client.15921 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613025de4a0 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c600bcf70 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c64005780 con 0x56130255b370 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:12.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 20 client.15921 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025dec70 0x5613025e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 4 client.15921 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.396+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613024bdd70 con 0x5613025dec70 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c71534640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025dec70 0x5613025e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c71534640 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025dec70 0x5613025e1090 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4808d22956b36c server_cookie=27f01933684d4fa7 in_seq=0 out_seq=0 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c54ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c54ff9640 1 -- 172.21.15.28:0/2393056105 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613024bdd70 con 0x5613025dec70 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c54ff9640 10 client.15921 handle_command_reply: tid=0 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 client.15921 shutdown 2024-09-19T09:34:12.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 msgr2=0x56130255f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1c64010b40 tx=0x7f1c64004ab0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 shutdown_connections 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561302551b40 0x56130255f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5613025dec70 0x5613025e1090 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c54ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56130255b370 0x56130255f870 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 --2- 172.21.15.28:0/2393056105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613025505d0 0x56130255edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.397+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 >> 172.21.15.28:0/2393056105 conn(0x561302519310 msgr2=0x561302559870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 shutdown_connections 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c75b71740 1 -- 172.21.15.28:0/2393056105 wait complete. 2024-09-19T09:34:12.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c75b71740 20 client.15921 trim_cache size 0 max 16384 2024-09-19T09:34:12.420 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:12.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 msgr2=0x7f1c580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 0x7f1c580a5780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400afd0 tx=0x7f1c64005c60 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 msgr2=0x7f1c6c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1c60005ab0 tx=0x7f1c600042f0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 shutdown_connections 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1c580a32c0 0x7f1c580a5780 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c6c101270 0x7f1c6c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c6c10e470 0x7f1c6c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 --2- 172.21.15.28:0/1897264128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c6c1008c0 0x7f1c6c19a6a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 >> 172.21.15.28:0/1897264128 conn(0x7f1c6c0fc7d0 msgr2=0x7f1c6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:12.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.398+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 shutdown_connections 2024-09-19T09:34:12.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:12.399+0000 7f1c73798640 1 -- 172.21.15.28:0/1897264128 wait complete. 2024-09-19T09:34:12.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:12 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1875521984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:13.331 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:13 smithi028 ceph-mon[30972]: pgmap v384: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-19T09:34:13.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:13 smithi078 ceph-mon[35619]: pgmap v384: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-19T09:34:13.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:13 smithi038 ceph-mon[35271]: pgmap v384: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/2018213280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c8101df0 msgr2=0x7ff7c81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/2018213280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c8101df0 0x7ff7c81021d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009ca0 tx=0x7ff7c402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/2018213280 shutdown_connections 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/2018213280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8106f20 0x7ff7c810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/2018213280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c81027a0 0x7ff7c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/2018213280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c8101df0 0x7ff7c81021d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/2018213280 >> 172.21.15.28:0/2018213280 conn(0x7ff7c80fd8f0 msgr2=0x7ff7c80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/2018213280 shutdown_connections 2024-09-19T09:34:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/2018213280 wait complete. 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 Processor -- start 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.562+0000 7ff7cfe4a640 1 -- start start 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 0x7ff7c81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c8106f20 0x7ff7c819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8111fa0 con 0x7ff7c81027a0 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8111e20 con 0x7ff7c8101df0 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8112120 con 0x7ff7c8106f20 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39348/0 (socket says 172.21.15.28:39348) 2024-09-19T09:34:13.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/245286237 learned_addr learned my addr 172.21.15.28:0/245286237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 0x7ff7c81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c8106f20 0x7ff7c819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/245286237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c8106f20 msgr2=0x7ff7c819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c8106f20 0x7ff7c819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/245286237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 msgr2=0x7ff7c81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 0x7ff7c81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/245286237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c819d450 con 0x7ff7c81027a0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 0x7ff7c81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b4a0 tx=0x7ff7bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7baffd640 1 -- 172.21.15.28:0/245286237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc014070 con 0x7ff7c81027a0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7baffd640 1 -- 172.21.15.28:0/245286237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7bc010af0 con 0x7ff7c81027a0 2024-09-19T09:34:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7baffd640 1 -- 172.21.15.28:0/245286237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc017d10 con 0x7ff7c81027a0 2024-09-19T09:34:13.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.563+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c819d740 con 0x7ff7c81027a0 2024-09-19T09:34:13.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.564+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81a5020 con 0x7ff7c81027a0 2024-09-19T09:34:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.566+0000 7ff7d2223740 1 Processor -- start 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.566+0000 7ff7baffd640 1 -- 172.21.15.28:0/245286237 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff7bc017070 con 0x7ff7c81027a0 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.566+0000 7ff7d2223740 1 -- start start 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.566+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 0x559ac94ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.566+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94f8240 0x559ac94fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7d2223740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559ac9452620 con 0x559ac94ed980 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7d2223740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ac93f9e50 con 0x559ac94ed060 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7d2223740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ac944e5e0 con 0x559ac94f8240 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39352/0 (socket says 172.21.15.28:39352) 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/2366387427 learned_addr learned my addr 172.21.15.28:0/2366387427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94f8240 0x559ac94fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 0x559ac94ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7baffd640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 0x7ff7b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/2366387427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94f8240 msgr2=0x559ac94fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94f8240 0x559ac94fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/2366387427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 msgr2=0x559ac94ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 0x559ac94ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/2366387427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac93ead40 con 0x559ac94ed980 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 0x559ac94ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 0x7ff7b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7baffd640 1 -- 172.21.15.28:0/245286237 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff7bc0cecf0 con 0x7ff7c81027a0 2024-09-19T09:34:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.567+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4000f80 tx=0x7ff7c4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39860a4c382d20c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7b3fff640 1 -- 172.21.15.28:0/2366387427 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4003970 con 0x559ac94ed980 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 0x7ff7b40a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000bce0 tx=0x7ff7c00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7b3fff640 1 -- 172.21.15.28:0/2366387427 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4036070 con 0x559ac94ed980 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7b3fff640 1 -- 172.21.15.28:0/2366387427 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40043c0 con 0x559ac94ed980 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7d2223740 1 -- 172.21.15.28:0/2366387427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 msgr2=0x559ac94f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.568+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4000f80 tx=0x7ff7c4009880 comp rx=0 tx=0).stop 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 -- 172.21.15.28:0/2366387427 shutdown_connections 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94f8240 0x559ac94fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94f7d00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2366387427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed060 0x559ac94ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 -- 172.21.15.28:0/2366387427 >> 172.21.15.28:0/2366387427 conn(0x559ac94e95f0 msgr2=0x559ac94e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 -- 172.21.15.28:0/2366387427 shutdown_connections 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 -- 172.21.15.28:0/2366387427 wait complete. 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 -- start start 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 Processor -- start 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac94f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed980 0x559ac94f5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.569+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7d2223740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559ac94fae10 con 0x559ac94f8240 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7d2223740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559ac93daf10 con 0x559ac94ed980 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7d2223740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ac94faf90 con 0x559ac94ed060 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7d2223740 10 client.0 _ll_register_callbacks cb 0x559ac93fedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39358/0 (socket says 172.21.15.28:39358) 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 -- 172.21.15.28:0/1466685672 learned_addr learned my addr 172.21.15.28:0/1466685672 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed980 0x559ac94f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac94f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 -- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 msgr2=0x559ac94f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac94f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 -- 172.21.15.28:0/1466685672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed980 msgr2=0x559ac94f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed980 0x559ac94f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 -- 172.21.15.28:0/1466685672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac9161580 con 0x559ac94f8240 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00126d0 tx=0x7ff7c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48e1c8769b162a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001e920 con 0x559ac94f8240 2024-09-19T09:34:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff7c001eac0 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.570+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c4036070 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7c001ecc0 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 10 client.15933 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ac922b960 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4036070 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7c00275c0 con 0x559ac94f8240 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 10 client.15933 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 10 client.15933 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 20 client.15933 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9507d40 0x559ac950a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 4 client.15933 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:13.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac945fb90 con 0x559ac9507d40 2024-09-19T09:34:13.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.571+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9507d40 0x559ac950a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.572+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9507d40 0x559ac950a1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12bae1150b6ac9b server_cookie=be298594b75eac52 in_seq=0 out_seq=0 2024-09-19T09:34:13.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.572+0000 7ff7b17fa640 10 client.15933 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.572+0000 7ff7b17fa640 1 -- 172.21.15.28:0/1466685672 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ac945fb90 con 0x559ac9507d40 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.572+0000 7ff7b17fa640 10 client.15933 handle_command_reply: tid=0 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 client.15933 shutdown 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 msgr2=0x559ac94f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00126d0 tx=0x7ff7c0004730 comp rx=0 tx=0).stop 2024-09-19T09:34:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 shutdown_connections 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94ed980 0x559ac94f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9507d40 0x559ac950a1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94f8240 0x559ac94f5b90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 --2- 172.21.15.28:0/1466685672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac94f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7b17fa640 0 client.15933 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 >> 172.21.15.28:0/1466685672 conn(0x559ac94e95f0 msgr2=0x559ac94ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 shutdown_connections 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 1 -- 172.21.15.28:0/1466685672 wait complete. 2024-09-19T09:34:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.573+0000 7ff7d2223740 20 client.15933 trim_cache size 0 max 16384 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 Processor -- start 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 -- start start 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed980 0x559ac9510040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81ad540 con 0x559ac94ed060 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81af080 con 0x559ac94f8240 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7d2223740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81adc10 con 0x559ac94ed980 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7cdbe6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7cdbe6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39368/0 (socket says 172.21.15.28:39368) 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/3119083526 learned_addr learned my addr 172.21.15.28:0/3119083526 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed980 0x559ac9510040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.590+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/3119083526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed980 msgr2=0x559ac9510040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed980 0x559ac9510040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/3119083526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 msgr2=0x559ac94f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 -- 172.21.15.28:0/3119083526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac94e9420 con 0x559ac94ed060 2024-09-19T09:34:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x559ac94ea450 tx=0x7ff7c4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b2ec340a9a4db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7b37fe640 1 -- 172.21.15.28:0/3119083526 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4031e10 con 0x559ac94ed060 2024-09-19T09:34:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7b37fe640 1 -- 172.21.15.28:0/3119083526 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4052070 con 0x559ac94ed060 2024-09-19T09:34:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.591+0000 7ff7b37fe640 1 -- 172.21.15.28:0/3119083526 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c402fe30 con 0x559ac94ed060 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- 172.21.15.28:0/3119083526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 msgr2=0x559ac950fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x559ac94ea450 tx=0x7ff7c4031a90 comp rx=0 tx=0).stop 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- 172.21.15.28:0/3119083526 shutdown_connections 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed980 0x559ac9510040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 --2- 172.21.15.28:0/3119083526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed060 0x559ac950fb00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- 172.21.15.28:0/3119083526 >> 172.21.15.28:0/3119083526 conn(0x559ac94e95f0 msgr2=0x559ac94f39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- 172.21.15.28:0/3119083526 shutdown_connections 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- 172.21.15.28:0/3119083526 wait complete. 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 -- start start 2024-09-19T09:34:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.592+0000 7ff7d2223740 1 Processor -- start 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac9500730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81ae9b0 con 0x559ac94ed980 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81ae2e0 con 0x559ac94f8240 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81ab520 con 0x559ac94ed060 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7d2223740 10 client.0 _ll_register_callbacks cb 0x7ff7c000bd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39378/0 (socket says 172.21.15.28:39378) 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/2850168320 learned_addr learned my addr 172.21.15.28:0/2850168320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac9500730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 msgr2=0x559ac9500730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac9500730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/2850168320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 msgr2=0x559ac94fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7cd3e5640 1 -- 172.21.15.28:0/2850168320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac94efed0 con 0x559ac94ed980 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.593+0000 7ff7ce3e7640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7cd3e5640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc004700 tx=0x7ff7bc0cf0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c090700f9324a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc01e040 con 0x559ac94ed980 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff7bc010ca0 con 0x559ac94ed980 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c4052070 con 0x559ac94ed980 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7bc09b340 con 0x559ac94ed980 2024-09-19T09:34:13.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 10 client.15939 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ac94ea280 con 0x559ac94ed980 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4052070 con 0x559ac94ed980 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7bc0ccce0 con 0x559ac94ed980 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 10 client.15939 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 20 client.15939 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9573d50 0x559ac9576170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 4 client.15939 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5bd3b1f", "release": true, "await": true}] 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.594+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5bd3b1f", "release": true, "await": true}) -- 0x559ac945a0a0 con 0x559ac9573d50 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.595+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9573d50 0x559ac9576170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.595+0000 7ff7b17fa640 10 client.15939 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.595+0000 7ff7cdbe6640 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9573d50 0x559ac9576170 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93630f404ba71d3 server_cookie=7040f0b9adf84ac in_seq=0 out_seq=0 2024-09-19T09:34:13.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7b17fa640 1 -- 172.21.15.28:0/2850168320 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x559ac945a0a0 con 0x559ac9573d50 2024-09-19T09:34:13.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7b17fa640 10 client.15939 handle_command_reply: tid=0 2024-09-19T09:34:13.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 client.15939 shutdown 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 msgr2=0x559ac94fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc004700 tx=0x7ff7bc0cf0d0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 shutdown_connections 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559ac94f8240 0x559ac94fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559ac9573d50 0x559ac9576170 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559ac94ed980 0x559ac94fca70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 --2- 172.21.15.28:0/2850168320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ac94ed060 0x559ac9500730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7b17fa640 0 client.15939 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 >> 172.21.15.28:0/2850168320 conn(0x559ac94e95f0 msgr2=0x559ac94eeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 shutdown_connections 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 1 -- 172.21.15.28:0/2850168320 wait complete. 2024-09-19T09:34:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.602+0000 7ff7d2223740 20 client.15939 trim_cache size 0 max 16384 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 8, 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "5bd3b1f": { 2024-09-19T09:34:13.606 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 8, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 140.3, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:13.607 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:13.608 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 msgr2=0x7ff7b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 0x7ff7b40a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000bce0 tx=0x7ff7c00046c0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 msgr2=0x7ff7c81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b4a0 tx=0x7ff7bc00b960 comp rx=0 tx=0).stop 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 shutdown_connections 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff7b40a32c0 0x7ff7b40a5780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7c8106f20 0x7ff7c819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7c81027a0 0x7ff7c81989e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 --2- 172.21.15.28:0/245286237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7c8101df0 0x7ff7c81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:13.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 >> 172.21.15.28:0/245286237 conn(0x7ff7c80fd8f0 msgr2=0x7ff7c80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:13.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 shutdown_connections 2024-09-19T09:34:13.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:13.603+0000 7ff7cfe4a640 1 -- 172.21.15.28:0/245286237 wait complete. 2024-09-19T09:34:13.637 INFO:tasks.workunit.client.0.smithi028.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-19T09:34:14.050 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:34:14.051 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:34:14.270 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5bd3b1f', total seconds elapsed: 24.2 2024-09-19T09:34:14.270 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:34:14.270 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:34:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:14.646 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether build environment is sane... yes 2024-09-19T09:34:14.662 INFO:tasks.workunit.client.0.smithi028.stdout:checking for gawk... gawk 2024-09-19T09:34:14.678 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether make sets $(MAKE)... yes 2024-09-19T09:34:14.706 INFO:tasks.workunit.client.0.smithi028.stdout:checking for gcc... gcc 2024-09-19T09:34:14.753 INFO:tasks.workunit.client.0.smithi028.stdout:checking for C compiler default output file name... a.out 2024-09-19T09:34:14.755 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether the C compiler works... yes 2024-09-19T09:34:14.757 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether we are cross compiling... no 2024-09-19T09:34:14.793 INFO:tasks.workunit.client.0.smithi028.stdout:checking for suffix of executables... 2024-09-19T09:34:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- 172.21.15.28:0/948591156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102570 msgr2=0x7eff501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 --2- 172.21.15.28:0/948591156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102570 0x7eff501029b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7eff40009900 tx=0x7eff4002a480 comp rx=0 tx=0).stop 2024-09-19T09:34:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- 172.21.15.28:0/948591156 shutdown_connections 2024-09-19T09:34:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 --2- 172.21.15.28:0/948591156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102ef0 0x7eff5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 --2- 172.21.15.28:0/948591156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102570 0x7eff501029b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 --2- 172.21.15.28:0/948591156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- 172.21.15.28:0/948591156 >> 172.21.15.28:0/948591156 conn(0x7eff500fc7d0 msgr2=0x7eff500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- 172.21.15.28:0/948591156 shutdown_connections 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- 172.21.15.28:0/948591156 wait complete. 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 Processor -- start 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.792+0000 7eff56cc7640 1 -- start start 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102570 0x7eff50198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff50110f80 con 0x7eff50102ef0 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff50110e00 con 0x7eff50102570 2024-09-19T09:34:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff56cc7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111100 con 0x7eff50100e20 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39394/0 (socket says 172.21.15.28:39394) 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 -- 172.21.15.28:0/558970940 learned_addr learned my addr 172.21.15.28:0/558970940 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff54a63640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff47fff640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102570 0x7eff50198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 -- 172.21.15.28:0/558970940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 msgr2=0x7eff50198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 -- 172.21.15.28:0/558970940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102570 msgr2=0x7eff50198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102570 0x7eff50198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 -- 172.21.15.28:0/558970940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5019d520 con 0x7eff50102ef0 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff54a63640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff55264640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eb10 tx=0x7eff4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.793+0000 7eff45ffb640 1 -- 172.21.15.28:0/558970940 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48004b20 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.794+0000 7eff45ffb640 1 -- 172.21.15.28:0/558970940 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff48004cc0 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.794+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5019d810 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.794+0000 7eff45ffb640 1 -- 172.21.15.28:0/558970940 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48010760 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.794+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a50f0 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.796+0000 7eff45ffb640 1 -- 172.21.15.28:0/558970940 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7eff48005020 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 Processor -- start 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 -- start start 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff45ffb640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 0x7eff380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff47fff640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 0x7eff380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff45ffb640 1 -- 172.21.15.28:0/558970940 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff480cfdd0 con 0x7eff50102ef0 2024-09-19T09:34:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562818820280 con 0x5628188baa00 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628187c7b10 con 0x5628188bb320 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff590a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56281881c240 con 0x5628188ca150 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39406/0 (socket says 172.21.15.28:39406) 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff54a63640 1 -- 172.21.15.28:0/1152506727 learned_addr learned my addr 172.21.15.28:0/1152506727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff55264640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.797+0000 7eff47fff640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 -- 172.21.15.28:0/1152506727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 msgr2=0x5628188cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 -- 172.21.15.28:0/1152506727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 msgr2=0x5628188c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 -- 172.21.15.28:0/1152506727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628187d41f0 con 0x5628188baa00 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff47fff640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff47fff640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 0x7eff380a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff40009230 tx=0x7eff40005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff54a63640 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0115c0 tx=0x7eff4c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a450e514e4dbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff36ffd640 1 -- 172.21.15.28:0/1152506727 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01e070 con 0x5628188baa00 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff36ffd640 1 -- 172.21.15.28:0/1152506727 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c00fa20 con 0x5628188baa00 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.798+0000 7eff36ffd640 1 -- 172.21.15.28:0/1152506727 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c019b00 con 0x5628188baa00 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- 172.21.15.28:0/1152506727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 msgr2=0x5628188bade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0115c0 tx=0x7eff4c0119c0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- 172.21.15.28:0/1152506727 shutdown_connections 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 --2- 172.21.15.28:0/1152506727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188bade0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- 172.21.15.28:0/1152506727 >> 172.21.15.28:0/1152506727 conn(0x5628188b70f0 msgr2=0x5628188b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- 172.21.15.28:0/1152506727 shutdown_connections 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- 172.21.15.28:0/1152506727 wait complete. 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 -- start start 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.799+0000 7eff590a0740 1 Processor -- start 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 0x5628188c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 0x5628188c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188ca150 0x5628188c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5628188ccd20 con 0x5628188bb320 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5628187daf10 con 0x5628188ca150 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628188ccea0 con 0x5628188baa00 2024-09-19T09:34:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff590a0740 10 client.0 _ll_register_callbacks cb 0x5628187a3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 0x5628188c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 0x5628188c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39416/0 (socket says 172.21.15.28:39416) 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff47fff640 1 -- 172.21.15.28:0/972162347 learned_addr learned my addr 172.21.15.28:0/972162347 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 0x5628188c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff55264640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188ca150 0x5628188c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 -- 172.21.15.28:0/972162347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188ca150 msgr2=0x5628188c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188ca150 0x5628188c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 -- 172.21.15.28:0/972162347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 msgr2=0x5628188c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 0x5628188c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.800+0000 7eff54a63640 1 -- 172.21.15.28:0/972162347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628185916a0 con 0x5628188baa00 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff54a63640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 0x5628188c7020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff4c01c040 tx=0x7eff4c019510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e880a7410ae1b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01e040 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4c00fa20 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff4c00f6c0 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff4c004040 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff590a0740 10 client.25513 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628188d8960 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.801+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c00fa20 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff4c021070 con 0x5628188baa00 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 10 client.25513 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 10 client.25513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 20 client.25513 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188d9d70 0x5628188dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 4 client.25513 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56281882d7f0 con 0x5628188d9d70 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff47fff640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188d9d70 0x5628188dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff47fff640 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188d9d70 0x5628188dc160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=256941b3ad0c820f server_cookie=b36dedb69e6ddf7e in_seq=0 out_seq=0 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.802+0000 7eff2ffff640 10 client.25513 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff2ffff640 1 -- 172.21.15.28:0/972162347 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56281882d7f0 con 0x5628188d9d70 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff2ffff640 10 client.25513 handle_command_reply: tid=0 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 client.25513 shutdown 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 msgr2=0x5628188c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 0x5628188c7020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff4c01c040 tx=0x7eff4c019510 comp rx=0 tx=0).stop 2024-09-19T09:34:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 shutdown_connections 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188ca150 0x5628188c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188d9d70 0x5628188dc160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188bb320 0x5628188c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 --2- 172.21.15.28:0/972162347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188baa00 0x5628188c7020 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff2ffff640 0 client.25513 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 >> 172.21.15.28:0/972162347 conn(0x5628188b70f0 msgr2=0x5628188bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 shutdown_connections 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.803+0000 7eff590a0740 1 -- 172.21.15.28:0/972162347 wait complete. 2024-09-19T09:34:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.804+0000 7eff590a0740 20 client.25513 trim_cache size 0 max 16384 2024-09-19T09:34:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 Processor -- start 2024-09-19T09:34:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 -- start start 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff501ab2d0 con 0x5628188baa00 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff501ace10 con 0x5628188bb320 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff590a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff501ab9a0 con 0x5628188ca150 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39428/0 (socket says 172.21.15.28:39428) 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff55264640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff54a63640 1 -- 172.21.15.28:0/1926685321 learned_addr learned my addr 172.21.15.28:0/1926685321 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff47fff640 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff55264640 1 -- 172.21.15.28:0/1926685321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 msgr2=0x5628188e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.822+0000 7eff55264640 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff55264640 1 -- 172.21.15.28:0/1926685321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 msgr2=0x5628188c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff55264640 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff55264640 1 -- 172.21.15.28:0/1926685321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628188c9620 con 0x5628188ca150 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff55264640 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188e1ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff480cd8a0 tx=0x7eff48098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f141fb3a80c62c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff367fc640 1 -- 172.21.15.28:0/1926685321 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4809c3c0 con 0x5628188ca150 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff367fc640 1 -- 172.21.15.28:0/1926685321 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff480a0770 con 0x5628188ca150 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.823+0000 7eff367fc640 1 -- 172.21.15.28:0/1926685321 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480a4e00 con 0x5628188ca150 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- 172.21.15.28:0/1926685321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 msgr2=0x5628188e1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188e1ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff480cd8a0 tx=0x7eff48098040 comp rx=0 tx=0).stop 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- 172.21.15.28:0/1926685321 shutdown_connections 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188e1ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 --2- 172.21.15.28:0/1926685321 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- 172.21.15.28:0/1926685321 >> 172.21.15.28:0/1926685321 conn(0x5628188b70f0 msgr2=0x5628188b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- 172.21.15.28:0/1926685321 shutdown_connections 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- 172.21.15.28:0/1926685321 wait complete. 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 -- start start 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.824+0000 7eff590a0740 1 Processor -- start 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff501ac740 con 0x5628188baa00 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff501ac070 con 0x5628188bb320 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff501a92d0 con 0x5628188ca150 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff590a0740 10 client.0 _ll_register_callbacks cb 0x7eff4c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39442/0 (socket says 172.21.15.28:39442) 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 -- 172.21.15.28:0/2500039288 learned_addr learned my addr 172.21.15.28:0/2500039288 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff47fff640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff55264640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 -- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 msgr2=0x5628188ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 -- 172.21.15.28:0/2500039288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 msgr2=0x5628188ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff54a63640 1 -- 172.21.15.28:0/2500039288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562818943b60 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.825+0000 7eff47fff640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff54a63640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009230 tx=0x7eff4c019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0176497df40a459 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01e040 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7eff4c00f540 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff480a0770 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff4c00f780 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff590a0740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.826+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628188bd7a0 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c027070 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff4c021070 con 0x5628188baa00 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 20 client.15957 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188bdef0 0x5628188c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 4 client.15957 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7eff480a0770 con 0x5628188bdef0 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.827+0000 7eff47fff640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188bdef0 0x5628188c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.828+0000 7eff47fff640 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188bdef0 0x5628188c0310 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785ba1822e243aed server_cookie=1bcd3cd88f76be40 in_seq=0 out_seq=0 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.828+0000 7eff2ffff640 10 client.15957 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.828+0000 7eff2ffff640 1 -- 172.21.15.28:0/2500039288 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7eff480a0770 con 0x5628188bdef0 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.828+0000 7eff2ffff640 10 client.15957 handle_command_reply: tid=0 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 client.15957 shutdown 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 msgr2=0x5628188ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009230 tx=0x7eff4c019410 comp rx=0 tx=0).stop 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 shutdown_connections 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5628188bb320 0x5628188ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5628188bdef0 0x5628188c0310 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff2ffff640 0 client.15957 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628188ca150 0x5628188ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 --2- 172.21.15.28:0/2500039288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5628188baa00 0x5628188ce0b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 >> 172.21.15.28:0/2500039288 conn(0x5628188b70f0 msgr2=0x5628188d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 shutdown_connections 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 1 -- 172.21.15.28:0/2500039288 wait complete. 2024-09-19T09:34:14.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.829+0000 7eff590a0740 20 client.15957 trim_cache size 0 max 16384 2024-09-19T09:34:14.841 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2024-09-19T09:34:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "51dd7015-8e9a-47d2-8ccc-78ee0e3ef4b2", 2024-09-19T09:34:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2024-09-19T09:34:14.841 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 msgr2=0x7eff380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 0x7eff380a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff40009230 tx=0x7eff40005e50 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 msgr2=0x7eff5019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eb10 tx=0x7eff4800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 shutdown_connections 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7eff380a32c0 0x7eff380a5780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff50102570 0x7eff50198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff50102ef0 0x7eff5019ce40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 --2- 172.21.15.28:0/558970940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff50100e20 0x7eff50198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 >> 172.21.15.28:0/558970940 conn(0x7eff500fc7d0 msgr2=0x7eff5006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:14.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 shutdown_connections 2024-09-19T09:34:14.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:14.830+0000 7eff56cc7640 1 -- 172.21.15.28:0/558970940 wait complete. 2024-09-19T09:34:14.843 INFO:tasks.workunit.client.0.smithi028.stdout:checking for suffix of object files... o 2024-09-19T09:34:14.874 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether we are using the GNU C compiler... yes 2024-09-19T09:34:14.916 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether gcc accepts -g... yes 2024-09-19T09:34:15.125 INFO:tasks.workunit.client.0.smithi028.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-19T09:34:15.189 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:15.208 INFO:tasks.workunit.client.0.smithi028.stdout:checking for style of include used by make... GNU 2024-09-19T09:34:15.300 INFO:tasks.workunit.client.0.smithi028.stdout:checking dependency style of gcc... gcc3 2024-09-19T09:34:15.300 INFO:tasks.workunit.client.0.smithi028.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-19T09:34:15.306 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:15 smithi028 ceph-mon[30972]: pgmap v385: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T09:34:15.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:15 smithi078 ceph-mon[35619]: pgmap v385: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T09:34:15.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:15 smithi038 ceph-mon[35271]: pgmap v385: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T09:34:15.467 INFO:tasks.workunit.client.0.smithi028.stdout:checking how to run the C preprocessor... gcc -E 2024-09-19T09:34:15.508 INFO:tasks.workunit.client.0.smithi028.stdout:checking for egrep... grep -E 2024-09-19T09:34:15.517 INFO:tasks.workunit.client.0.smithi028.stdout:checking for AIX... no 2024-09-19T09:34:15.591 INFO:tasks.workunit.client.0.smithi028.stdout:checking for library containing strerror... none required 2024-09-19T09:34:15.703 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:34:15.801 INFO:tasks.workunit.client.0.smithi028.stdout:checking for ANSI C header files... yes 2024-09-19T09:34:15.847 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-19T09:34:15.894 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/types.h... yes 2024-09-19T09:34:15.950 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/stat.h... yes 2024-09-19T09:34:16.000 INFO:tasks.workunit.client.0.smithi028.stdout:checking for stdlib.h... yes 2024-09-19T09:34:16.050 INFO:tasks.workunit.client.0.smithi028.stdout:checking for string.h... yes 2024-09-19T09:34:16.138 INFO:tasks.workunit.client.0.smithi028.stdout:checking for memory.h... yes 2024-09-19T09:34:16.192 INFO:tasks.workunit.client.0.smithi028.stdout:checking for strings.h... yes 2024-09-19T09:34:16.243 INFO:tasks.workunit.client.0.smithi028.stdout:checking for inttypes.h... yes 2024-09-19T09:34:16.302 INFO:tasks.workunit.client.0.smithi028.stdout:checking for stdint.h... yes 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 -- 172.21.15.28:0/3466948353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4103660 msgr2=0x7efdc410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 --2- 172.21.15.28:0/3466948353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4103660 0x7efdc410bb50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009900 tx=0x7efdb802a480 comp rx=0 tx=0).stop 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 -- 172.21.15.28:0/3466948353 shutdown_connections 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 --2- 172.21.15.28:0/3466948353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc410c170 0x7efdc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 --2- 172.21.15.28:0/3466948353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4103660 0x7efdc410bb50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 --2- 172.21.15.28:0/3466948353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4102d40 0x7efdc4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 -- 172.21.15.28:0/3466948353 >> 172.21.15.28:0/3466948353 conn(0x7efdc40fc7a0 msgr2=0x7efdc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 -- 172.21.15.28:0/3466948353 shutdown_connections 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.276+0000 7efdcc698640 1 -- 172.21.15.28:0/3466948353 wait complete. 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 Processor -- start 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 -- start start 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 0x7efdc419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 0x7efdc419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4111f70 con 0x7efdc4102d40 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4111df0 con 0x7efdc4103660 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcc698640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41120f0 con 0x7efdc410c170 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39458/0 (socket says 172.21.15.28:39458) 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 -- 172.21.15.28:0/4274450775 learned_addr learned my addr 172.21.15.28:0/4274450775 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdc9c33640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 0x7efdc419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdcac35640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 0x7efdc419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 -- 172.21.15.28:0/4274450775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 msgr2=0x7efdc419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 0x7efdc419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 -- 172.21.15.28:0/4274450775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 msgr2=0x7efdc419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 0x7efdc419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.277+0000 7efdca434640 1 -- 172.21.15.28:0/4274450775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc419f5c0 con 0x7efdc4102d40 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdca434640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e9f0 tx=0x7efdc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc000cd80 con 0x7efdc4102d40 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc419f8b0 con 0x7efdc4102d40 2024-09-19T09:34:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc0010910 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00188f0 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.278+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc41a7190 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.279+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc4103120 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.280+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7efdc0010ac0 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.281+0000 7efdb77fe640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 0x7efdb00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.281+0000 7efdc9c33640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 0x7efdb00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.281+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7efdc00cfc40 con 0x7efdc4102d40 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.281+0000 7efdc9c33640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 0x7efdb00a5780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.285+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efdc009c340 con 0x7efdc4102d40 2024-09-19T09:34:16.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:16 smithi028 ceph-mon[30972]: scrub queued for path: / 2024-09-19T09:34:16.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:16 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:16 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:34:16.309 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:16 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.344 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:16.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:16 smithi078 ceph-mon[35619]: scrub queued for path: / 2024-09-19T09:34:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:16 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:16 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:34:16.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:16 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.358 INFO:tasks.workunit.client.0.smithi028.stdout:checking for unistd.h... yes 2024-09-19T09:34:16.371 INFO:tasks.workunit.client.0.smithi028.stdout:checking for memory.h... (cached) yes 2024-09-19T09:34:16.377 INFO:tasks.workunit.client.0.smithi028.stdout:checking for strings.h... (cached) yes 2024-09-19T09:34:16.382 INFO:tasks.workunit.client.0.smithi028.stdout:checking for unistd.h... (cached) yes 2024-09-19T09:34:16.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:16 smithi038 ceph-mon[35271]: scrub queued for path: / 2024-09-19T09:34:16.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:16 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:16 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:34:16.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:16 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:34:16.443 INFO:tasks.workunit.client.0.smithi028.stdout:checking locale.h usability... yes 2024-09-19T09:34:16.462 INFO:tasks.workunit.client.0.smithi028.stdout:checking locale.h presence... yes 2024-09-19T09:34:16.462 INFO:tasks.workunit.client.0.smithi028.stdout:checking for locale.h... yes 2024-09-19T09:34:16.477 INFO:tasks.workunit.client.0.smithi028.stdout:checking for stdlib.h... (cached) yes 2024-09-19T09:34:16.482 INFO:tasks.workunit.client.0.smithi028.stdout:checking for string.h... (cached) yes 2024-09-19T09:34:16.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.512+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdc41a7420 con 0x7efdc4102d40 2024-09-19T09:34:16.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.513+0000 7efdb77fe640 1 -- 172.21.15.28:0/4274450775 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7efdc00a0640 con 0x7efdc4102d40 2024-09-19T09:34:16.515 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:34:16.515 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:34:16.516 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.515+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 msgr2=0x7efdb00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.515+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 0x7efdb00a5780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb80058d0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 msgr2=0x7efdc419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e9f0 tx=0x7efdc000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 shutdown_connections 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a32c0 0x7efdb00a5780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4103660 0x7efdc419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc410c170 0x7efdc419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 --2- 172.21.15.28:0/4274450775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4102d40 0x7efdc419a610 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 >> 172.21.15.28:0/4274450775 conn(0x7efdc40fc7a0 msgr2=0x7efdc41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 shutdown_connections 2024-09-19T09:34:16.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.516+0000 7efdcc698640 1 -- 172.21.15.28:0/4274450775 wait complete. 2024-09-19T09:34:16.639 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/socket.h usability... yes 2024-09-19T09:34:16.663 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/socket.h presence... yes 2024-09-19T09:34:16.667 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/socket.h... yes 2024-09-19T09:34:16.722 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/time.h usability... yes 2024-09-19T09:34:16.742 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/time.h presence... yes 2024-09-19T09:34:16.743 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/time.h... yes 2024-09-19T09:34:16.798 INFO:tasks.workunit.client.0.smithi028.stdout:checking fcntl.h usability... yes 2024-09-19T09:34:16.818 INFO:tasks.workunit.client.0.smithi028.stdout:checking fcntl.h presence... yes 2024-09-19T09:34:16.818 INFO:tasks.workunit.client.0.smithi028.stdout:checking for fcntl.h... yes 2024-09-19T09:34:16.893 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/fcntl.h usability... yes 2024-09-19T09:34:16.913 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/fcntl.h presence... yes 2024-09-19T09:34:16.914 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/fcntl.h... yes 2024-09-19T09:34:16.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/2421665273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc102ef0 msgr2=0x7f5acc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/2421665273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc102ef0 0x7f5acc10da40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402a480 comp rx=0 tx=0).stop 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/2421665273 shutdown_connections 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/2421665273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/2421665273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc102ef0 0x7f5acc10da40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/2421665273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc100e20 0x7f5acc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/2421665273 >> 172.21.15.28:0/2421665273 conn(0x7f5acc0fc7d0 msgr2=0x7f5acc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/2421665273 shutdown_connections 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/2421665273 wait complete. 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 Processor -- start 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.945+0000 7f5ad2ab0640 1 -- start start 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc102ef0 0x7f5acc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1120a0 con 0x7f5acc100e20 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc111f20 con 0x7f5acc102570 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc112220 con 0x7f5acc102ef0 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39470/0 (socket says 172.21.15.28:39470) 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 -- 172.21.15.28:0/1260298322 learned_addr learned my addr 172.21.15.28:0/1260298322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5acbfff640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad104d640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc102ef0 0x7f5acc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 -- 172.21.15.28:0/1260298322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc102ef0 msgr2=0x7f5acc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc102ef0 0x7f5acc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 -- 172.21.15.28:0/1260298322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 msgr2=0x7f5acc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 -- 172.21.15.28:0/1260298322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc19d3b0 con 0x7f5acc100e20 2024-09-19T09:34:16.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5acbfff640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad084c640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cce0 tx=0x7f5abc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ac9ffb640 1 -- 172.21.15.28:0/1260298322 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00f650 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ac9ffb640 1 -- 172.21.15.28:0/1260298322 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc00fcd0 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ac9ffb640 1 -- 172.21.15.28:0/1260298322 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc018c70 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.946+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc19d6a0 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.947+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1a4f80 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.949+0000 7f5ac9ffb640 1 -- 172.21.15.28:0/1260298322 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5abc00f7f0 con 0x7f5acc100e20 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.949+0000 7f5ad4e89740 1 Processor -- start 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.949+0000 7f5ad4e89740 1 -- start start 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e36550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e38e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0d8d2e0 con 0x55c8b0e36a90 2024-09-19T09:34:16.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0d34ce0 con 0x55c8b0e27ce0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0d892a0 con 0x55c8b0e273c0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad104d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e38e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad104d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e277a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45174/0 (socket says 172.21.15.28:45174) 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad104d640 1 -- 172.21.15.28:0/1164557964 learned_addr learned my addr 172.21.15.28:0/1164557964 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ac9ffb640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 0x7f5ab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad084c640 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e36550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 -- 172.21.15.28:0/1164557964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 msgr2=0x55c8b0e277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad104d640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 0x7f5ab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 -- 172.21.15.28:0/1164557964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 msgr2=0x55c8b0e36550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e36550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5acbfff640 1 -- 172.21.15.28:0/1164557964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b0d19440 con 0x55c8b0e36a90 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ad084c640 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e36550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.950+0000 7f5ac9ffb640 1 -- 172.21.15.28:0/1260298322 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5abc0cfea0 con 0x7f5acc100e20 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.951+0000 7f5acbfff640 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e38e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009b60 tx=0x7f5ab400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c99212700771c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.951+0000 7f5ac2ffd640 1 -- 172.21.15.28:0/1164557964 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4016070 con 0x55c8b0e36a90 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.951+0000 7f5ad104d640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 0x7f5ab80a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005c40 tx=0x7f5ac402a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.951+0000 7f5ac2ffd640 1 -- 172.21.15.28:0/1164557964 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab4009e50 con 0x55c8b0e36a90 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.951+0000 7f5ac2ffd640 1 -- 172.21.15.28:0/1164557964 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4011470 con 0x55c8b0e36a90 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- 172.21.15.28:0/1164557964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 msgr2=0x55c8b0e38e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e38e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009b60 tx=0x7f5ab400e6c0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- 172.21.15.28:0/1164557964 shutdown_connections 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e36550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e38e80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/1164557964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- 172.21.15.28:0/1164557964 >> 172.21.15.28:0/1164557964 conn(0x55c8b0e23a30 msgr2=0x55c8b0e23e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- 172.21.15.28:0/1164557964 shutdown_connections 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- 172.21.15.28:0/1164557964 wait complete. 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 -- start start 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.952+0000 7f5ad4e89740 1 Processor -- start 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e33660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0e39600 con 0x55c8b0e36a90 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0d48050 con 0x55c8b0e273c0 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c8b0e39780 con 0x55c8b0e27ce0 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad4e89740 10 client.0 _ll_register_callbacks cb 0x55c8b0d88160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39484/0 (socket says 172.21.15.28:39484) 2024-09-19T09:34:16.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 -- 172.21.15.28:0/2248326660 learned_addr learned my addr 172.21.15.28:0/2248326660 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad084c640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5acbfff640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e33660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 -- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 msgr2=0x55c8b0e33660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e33660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 -- 172.21.15.28:0/2248326660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 msgr2=0x55c8b0e363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 -- 172.21.15.28:0/2248326660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b0afe3c0 con 0x55c8b0e36a90 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad084c640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5ad104d640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009230 tx=0x7f5ac4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b676cb7b8879cc8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.953+0000 7f5aabfff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac404e020 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab4009e50 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5ac40035f0 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ac4035070 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8b0e45040 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab4009e50 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ac4031700 con 0x55c8b0e36a90 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 20 client.15975 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e46450 0x55c8b0e488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 4 client.15975 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.954+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8b0d9a850 con 0x55c8b0e46450 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5ad084c640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e46450 0x55c8b0e488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5ad084c640 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e46450 0x55c8b0e488c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4748fc9af45da586 server_cookie=a9fb3286ad7a29ed in_seq=0 out_seq=0 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5aabfff640 10 client.15975 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5aabfff640 1 -- 172.21.15.28:0/2248326660 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8b0d9a850 con 0x55c8b0e46450 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5aabfff640 10 client.15975 handle_command_reply: tid=0 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.955+0000 7f5ad4e89740 1 client.15975 shutdown 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 msgr2=0x55c8b0e33ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009230 tx=0x7f5ac4009260 comp rx=0 tx=0).stop 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 shutdown_connections 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e46450 0x55c8b0e488c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e36a90 0x55c8b0e33ba0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e33660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/2248326660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5aabfff640 0 client.15975 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:16.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 >> 172.21.15.28:0/2248326660 conn(0x55c8b0e23a30 msgr2=0x55c8b0e296c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 shutdown_connections 2024-09-19T09:34:16.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 1 -- 172.21.15.28:0/2248326660 wait complete. 2024-09-19T09:34:16.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.956+0000 7f5ad4e89740 20 client.15975 trim_cache size 0 max 16384 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.973+0000 7f5ad4e89740 1 Processor -- start 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.973+0000 7f5ad4e89740 1 -- start start 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.973+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.973+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.973+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e4e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1ab210 con 0x55c8b0e23a30 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1acd50 con 0x55c8b0e273c0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1ab8e0 con 0x55c8b0e27ce0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39488/0 (socket says 172.21.15.28:39488) 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 -- 172.21.15.28:0/3691435939 learned_addr learned my addr 172.21.15.28:0/3691435939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad104d640 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e4e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5acbfff640 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 -- 172.21.15.28:0/3691435939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 msgr2=0x55c8b0e4e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e4e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 -- 172.21.15.28:0/3691435939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 msgr2=0x55c8b0e2ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e2ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 -- 172.21.15.28:0/3691435939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b0e36720 con 0x55c8b0e23a30 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5abc098e10 tx=0x7f5abc098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6cd52a068c28bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.974+0000 7f5ac27fc640 1 -- 172.21.15.28:0/3691435939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc085030 con 0x55c8b0e23a30 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ac27fc640 1 -- 172.21.15.28:0/3691435939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc09cb90 con 0x55c8b0e23a30 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ac27fc640 1 -- 172.21.15.28:0/3691435939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0cda80 con 0x55c8b0e23a30 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3691435939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 msgr2=0x55c8b0e428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5abc098e10 tx=0x7f5abc098040 comp rx=0 tx=0).stop 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3691435939 shutdown_connections 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e273c0 0x55c8b0e2ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e4e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3691435939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e23a30 0x55c8b0e428d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3691435939 >> 172.21.15.28:0/3691435939 conn(0x55c8b0e33d00 msgr2=0x55c8b0e2e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3691435939 shutdown_connections 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.975+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3691435939 wait complete. 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 -- start start 2024-09-19T09:34:16.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 Processor -- start 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 0x55c8b0e3efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e3a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1ac680 con 0x55c8b0e27ce0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1abfb0 con 0x55c8b0e23a30 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a91f0 con 0x55c8b0e273c0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad4e89740 10 client.0 _ll_register_callbacks cb 0x7f5ac4005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e3a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45208/0 (socket says 172.21.15.28:45208) 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5acbfff640 1 -- 172.21.15.28:0/3685812493 learned_addr learned my addr 172.21.15.28:0/3685812493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 0x55c8b0e3efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.976+0000 7f5ad104d640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e3a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 -- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 msgr2=0x55c8b0e3a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e3a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 -- 172.21.15.28:0/3685812493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 msgr2=0x55c8b0e3a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e3a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 -- 172.21.15.28:0/3685812493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b0e3ae90 con 0x55c8b0e23a30 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad104d640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e3a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad084c640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 0x55c8b0e3efe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c8b0e36660 tx=0x7f5abc0cd920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1837eb039d6f214 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5aabfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc085040 con 0x55c8b0e23a30 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc09cb90 con 0x55c8b0e23a30 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.977+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5abc0b5260 con 0x55c8b0e23a30 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5abc0a5100 con 0x55c8b0e23a30 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 10 client.25613 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:16.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8b0eb1010 con 0x55c8b0e23a30 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5abc09cb90 con 0x55c8b0e23a30 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5abc0d3040 con 0x55c8b0e23a30 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 10 client.25613 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 20 client.25613 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e36a90 0x55c8b0eb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 4 client.25613 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8b0d94d60 con 0x55c8b0e36a90 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.978+0000 7f5acbfff640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e36a90 0x55c8b0eb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5aabfff640 10 client.25613 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5acbfff640 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e36a90 0x55c8b0eb3710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b50246a67f8374 server_cookie=ed54b5f3f39c3a1c in_seq=0 out_seq=0 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5aabfff640 1 -- 172.21.15.28:0/3685812493 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8b0d94d60 con 0x55c8b0e36a90 2024-09-19T09:34:16.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5aabfff640 10 client.25613 handle_command_reply: tid=0 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 client.25613 shutdown 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 msgr2=0x55c8b0e3efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 0x55c8b0e3efe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c8b0e36660 tx=0x7f5abc0cd920 comp rx=0 tx=0).stop 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 shutdown_connections 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55c8b0e36a90 0x55c8b0eb3710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8b0e27ce0 0x55c8b0e3a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8b0e273c0 0x55c8b0e3a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5aabfff640 0 client.25613 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 --2- 172.21.15.28:0/3685812493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8b0e23a30 0x55c8b0e3efe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.979+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 >> 172.21.15.28:0/3685812493 conn(0x55c8b0e33d00 msgr2=0x55c8b0e22350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 shutdown_connections 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad4e89740 1 -- 172.21.15.28:0/3685812493 wait complete. 2024-09-19T09:34:16.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad4e89740 20 client.25613 trim_cache size 0 max 16384 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "51dd7015-8e9a-47d2-8ccc-78ee0e3ef4b2": { 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "51dd7015-8e9a-47d2-8ccc-78ee0e3ef4b2", 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive,force" 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:16.991 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 msgr2=0x7f5ab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 0x7f5ab80a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005c40 tx=0x7f5ac402a480 comp rx=0 tx=0).stop 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 msgr2=0x7f5acc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cce0 tx=0x7f5abc00a620 comp rx=0 tx=0).stop 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 shutdown_connections 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5ab80a3240 0x7f5ab80a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5acc102570 0x7f5acc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5acc102ef0 0x7f5acc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 --2- 172.21.15.28:0/1260298322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5acc100e20 0x7f5acc198400 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 >> 172.21.15.28:0/1260298322 conn(0x7f5acc0fc7d0 msgr2=0x7f5acc1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.980+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 shutdown_connections 2024-09-19T09:34:16.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:16.981+0000 7f5ad2ab0640 1 -- 172.21.15.28:0/1260298322 wait complete. 2024-09-19T09:34:17.132 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/param.h usability... yes 2024-09-19T09:34:17.155 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/param.h presence... yes 2024-09-19T09:34:17.155 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/param.h... yes 2024-09-19T09:34:17.207 INFO:tasks.workunit.client.0.smithi028.stdout:checking ioctl.h usability... no 2024-09-19T09:34:17.227 INFO:tasks.workunit.client.0.smithi028.stdout:checking ioctl.h presence... no 2024-09-19T09:34:17.228 INFO:tasks.workunit.client.0.smithi028.stdout:checking for ioctl.h... no 2024-09-19T09:34:17.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:17 smithi028 ceph-mon[30972]: pgmap v386: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T09:34:17.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:17 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4274450775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:17.279 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/ioctl.h usability... yes 2024-09-19T09:34:17.300 INFO:tasks.workunit.client.0.smithi028.stdout:checking sys/ioctl.h presence... yes 2024-09-19T09:34:17.301 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sys/ioctl.h... yes 2024-09-19T09:34:17.352 INFO:tasks.workunit.client.0.smithi028.stdout:checking return type of signal handlers... void 2024-09-19T09:34:17.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:17 smithi078 ceph-mon[35619]: pgmap v386: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T09:34:17.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:17 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4274450775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:17.397 INFO:tasks.workunit.client.0.smithi028.stdout:checking for size_t... yes 2024-09-19T09:34:17.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:17 smithi038 ceph-mon[35271]: pgmap v386: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T09:34:17.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:17 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4274450775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:17.441 INFO:tasks.workunit.client.0.smithi028.stdout:checking for dev_t... yes 2024-09-19T09:34:17.508 INFO:tasks.workunit.client.0.smithi028.stdout:checking for short... yes 2024-09-19T09:34:17.586 INFO:tasks.workunit.client.0.smithi028.stdout:checking size of short... 2 2024-09-19T09:34:17.601 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:34:17.602 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:34:17.633 INFO:tasks.workunit.client.0.smithi028.stdout:checking for int... yes 2024-09-19T09:34:17.718 INFO:tasks.workunit.client.0.smithi028.stdout:checking size of int... 4 2024-09-19T09:34:17.764 INFO:tasks.workunit.client.0.smithi028.stdout:checking for long... yes 2024-09-19T09:34:17.839 INFO:tasks.workunit.client.0.smithi028.stdout:checking size of long... 8 2024-09-19T09:34:17.891 INFO:tasks.workunit.client.0.smithi028.stdout:checking for long long... yes 2024-09-19T09:34:17.966 INFO:tasks.workunit.client.0.smithi028.stdout:checking size of long long... 8 2024-09-19T09:34:18.009 INFO:tasks.workunit.client.0.smithi028.stdout:checking for ssize_t... yes 2024-09-19T09:34:18.059 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sig_atomic_t... yes 2024-09-19T09:34:18.207 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether gcc needs -traditional... no 2024-09-19T09:34:18.248 INFO:tasks.workunit.client.0.smithi028.stdout:checking for an ANSI C-conforming const... yes 2024-09-19T09:34:18.275 INFO:tasks.workunit.client.0.smithi028.stdout:checking for inline... inline 2024-09-19T09:34:18.338 INFO:tasks.workunit.client.0.smithi028.stdout:checking for working alloca.h... yes 2024-09-19T09:34:18.366 INFO:tasks.cephfs.filesystem:scrub status for tag:51dd7015-8e9a-47d2-8ccc-78ee0e3ef4b2 - {'path': '/', 'tag': '51dd7015-8e9a-47d2-8ccc-78ee0e3ef4b2', 'options': 'recursive,force'} 2024-09-19T09:34:18.390 INFO:tasks.workunit.client.0.smithi028.stdout:checking for alloca... yes 2024-09-19T09:34:18.417 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:18.475 INFO:tasks.workunit.client.0.smithi028.stdout:checking for working memcmp... yes 2024-09-19T09:34:18.532 INFO:tasks.workunit.client.0.smithi028.stdout:checking for strftime... yes 2024-09-19T09:34:18.616 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-19T09:34:18.683 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether stat accepts an empty string... no 2024-09-19T09:34:18.744 INFO:tasks.workunit.client.0.smithi028.stdout:checking for vprintf... yes 2024-09-19T09:34:18.828 INFO:tasks.workunit.client.0.smithi028.stdout:checking for _doprnt... no 2024-09-19T09:34:18.850 INFO:tasks.workunit.client.0.smithi028.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-19T09:34:18.864 INFO:tasks.workunit.client.0.smithi028.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-19T09:34:18.939 INFO:tasks.workunit.client.0.smithi028.stdout:checking for the pthreads library -lpthreads... no 2024-09-19T09:34:19.013 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether pthreads work without any flags... yes 2024-09-19T09:34:19.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 -- 172.21.15.28:0/304055954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa4102d30 msgr2=0x7f8aa4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/304055954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa4102d30 0x7f8aa4103110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009930 tx=0x7f8aa002a420 comp rx=0 tx=0).stop 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 -- 172.21.15.28:0/304055954 shutdown_connections 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/304055954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa41082a0 0x7f8aa410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/304055954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/304055954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa4102d30 0x7f8aa4103110 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 -- 172.21.15.28:0/304055954 >> 172.21.15.28:0/304055954 conn(0x7f8aa40fc7d0 msgr2=0x7f8aa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 -- 172.21.15.28:0/304055954 shutdown_connections 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.011+0000 7f8aab4d7640 1 -- 172.21.15.28:0/304055954 wait complete. 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 Processor -- start 2024-09-19T09:34:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 -- start start 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa4102d30 0x7f8aa4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4112100 con 0x7f8aa41082a0 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4111f80 con 0x7f8aa4102d30 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aab4d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4112280 con 0x7f8aa41036e0 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aa8a72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39504/0 (socket says 172.21.15.28:39504) 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aa9a74640 1 -- 172.21.15.28:0/1189397294 learned_addr learned my addr 172.21.15.28:0/1189397294 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.012+0000 7f8aa9273640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa4102d30 0x7f8aa4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 -- 172.21.15.28:0/1189397294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa4102d30 msgr2=0x7f8aa4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa4102d30 0x7f8aa4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 -- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 msgr2=0x7f8aa4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 -- 172.21.15.28:0/1189397294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa41a6a80 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4100ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40697d0 tx=0x7f8a9400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8a9a7fc640 1 -- 172.21.15.28:0/1189397294 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400cc40 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa41a6d70 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8a9a7fc640 1 -- 172.21.15.28:0/1189397294 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9400cde0 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a7200 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.013+0000 7f8a9a7fc640 1 -- 172.21.15.28:0/1189397294 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004a00 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8a9a7fc640 1 -- 172.21.15.28:0/1189397294 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8a94004ba0 con 0x7f8aa41036e0 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 Processor -- start 2024-09-19T09:34:19.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 -- start start 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b915c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b922e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56001b87a160 con 0x56001b916170 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56001b821720 con 0x56001b915850 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56001b876120 con 0x56001b920a30 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aa9273640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aa9273640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39512/0 (socket says 172.21.15.28:39512) 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aa9273640 1 -- 172.21.15.28:0/3208766509 learned_addr learned my addr 172.21.15.28:0/3208766509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.016+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b922e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b915c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 -- 172.21.15.28:0/3208766509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 msgr2=0x56001b922e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b922e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 -- 172.21.15.28:0/3208766509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 msgr2=0x56001b915c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b915c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 -- 172.21.15.28:0/3208766509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56001b5ebba0 con 0x56001b916170 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b915c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8a9a7fc640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 0x7f8a900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 0x7f8a900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009930 tx=0x7f8aa0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212804bd8ffefc96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8a9a7fc640 1 -- 172.21.15.28:0/1189397294 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8a940cf800 con 0x7f8aa41036e0 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8a8f7fe640 1 -- 172.21.15.28:0/3208766509 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0035040 con 0x56001b916170 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8a8f7fe640 1 -- 172.21.15.28:0/3208766509 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0005520 con 0x56001b916170 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.017+0000 7f8a8f7fe640 1 -- 172.21.15.28:0/3208766509 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004e60 con 0x56001b916170 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 0x7f8a900a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0049d0 tx=0x7f8a9c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3208766509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 msgr2=0x56001b9204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009930 tx=0x7f8aa0005de0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3208766509 shutdown_connections 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b922e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b9204f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3208766509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b915c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3208766509 >> 172.21.15.28:0/3208766509 conn(0x56001b911de0 msgr2=0x56001b9121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3208766509 shutdown_connections 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.018+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3208766509 wait complete. 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 -- start start 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 Processor -- start 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 0x56001b91d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 0x56001b91de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 0x56001b91e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56001b923600 con 0x56001b920a30 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56001b802900 con 0x56001b916170 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56001b923780 con 0x56001b915850 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aad8b0740 10 client.0 _ll_register_callbacks cb 0x56001b800780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 0x56001b91e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 0x56001b91e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39520/0 (socket says 172.21.15.28:39520) 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aa9a74640 1 -- 172.21.15.28:0/3240793059 learned_addr learned my addr 172.21.15.28:0/3240793059 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 0x56001b91d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.019+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 0x56001b91de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 -- 172.21.15.28:0/3240793059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 msgr2=0x56001b91de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 0x56001b91de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 -- 172.21.15.28:0/3240793059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 msgr2=0x56001b91e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 0x56001b91e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 -- 172.21.15.28:0/3240793059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001b842c50 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 0x56001b91de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 0x56001b91d940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0031550 tx=0x7f8aa0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35077568a53d2366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8a8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0052060 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8aa00042d0 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa0005520 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8aa0035050 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aad8b0740 10 client.25552 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.020+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56001b923600 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0005520 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8aa00038c0 con 0x56001b915850 2024-09-19T09:34:19.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 10 client.25552 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 10 client.25552 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 20 client.25552 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b930460 0x56001b932850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 4 client.25552 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56001b887890 con 0x56001b930460 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b930460 0x56001b932850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8a8cff9640 10 client.25552 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:19.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.021+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b930460 0x56001b932850 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c99ad2c4837c86 server_cookie=2c4c728cecb7f47f in_seq=0 out_seq=0 2024-09-19T09:34:19.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3240793059 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56001b887890 con 0x56001b930460 2024-09-19T09:34:19.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8a8cff9640 10 client.25552 handle_command_reply: tid=0 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 client.25552 shutdown 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 msgr2=0x56001b91d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 0x56001b91d940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0031550 tx=0x7f8aa0005b50 comp rx=0 tx=0).stop 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 shutdown_connections 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b916170 0x56001b91de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b930460 0x56001b932850 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b920a30 0x56001b91e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3240793059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b915850 0x56001b91d940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8a8cff9640 0 client.25552 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 >> 172.21.15.28:0/3240793059 conn(0x56001b911de0 msgr2=0x56001b917b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 shutdown_connections 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.022+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3240793059 wait complete. 2024-09-19T09:34:19.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.023+0000 7f8aad8b0740 20 client.25552 trim_cache size 0 max 16384 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 Processor -- start 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 -- start start 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b9182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b937f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41af710 con 0x56001b916170 2024-09-19T09:34:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41b1250 con 0x56001b915850 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41afde0 con 0x56001b920a30 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aa8a72640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b937f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.040+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa8a72640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b937f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39536/0 (socket says 172.21.15.28:39536) 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9273640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b9182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45266/0 (socket says 172.21.15.28:45266) 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa8a72640 1 -- 172.21.15.28:0/957473762 learned_addr learned my addr 172.21.15.28:0/957473762 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 -- 172.21.15.28:0/957473762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 msgr2=0x56001b9182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b9182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 -- 172.21.15.28:0/957473762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 msgr2=0x56001b937f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b937f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 -- 172.21.15.28:0/957473762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56001b911c10 con 0x56001b920a30 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c012990 tx=0x7f8a9c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d046ef2a8282de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8a8effd640 1 -- 172.21.15.28:0/957473762 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0119c0 con 0x56001b920a30 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8a8effd640 1 -- 172.21.15.28:0/957473762 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c011b60 con 0x56001b920a30 2024-09-19T09:34:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.041+0000 7f8a8effd640 1 -- 172.21.15.28:0/957473762 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01f680 con 0x56001b920a30 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- 172.21.15.28:0/957473762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 msgr2=0x56001b938480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c012990 tx=0x7f8a9c01a000 comp rx=0 tx=0).stop 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- 172.21.15.28:0/957473762 shutdown_connections 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b920a30 0x56001b938480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b916170 0x56001b937f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/957473762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b915850 0x56001b9182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- 172.21.15.28:0/957473762 >> 172.21.15.28:0/957473762 conn(0x56001b911de0 msgr2=0x56001b91e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- 172.21.15.28:0/957473762 shutdown_connections 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- 172.21.15.28:0/957473762 wait complete. 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 -- start start 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.042+0000 7f8aad8b0740 1 Processor -- start 2024-09-19T09:34:19.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 0x56001b925810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b920a30 0x56001b992ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41b0b80 con 0x56001b915850 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41b04b0 con 0x56001b920a30 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ad710 con 0x56001b916170 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aad8b0740 10 client.0 _ll_register_callbacks cb 0x7f8a940d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa9273640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa9273640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39542/0 (socket says 172.21.15.28:39542) 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa9273640 1 -- 172.21.15.28:0/3497934462 learned_addr learned my addr 172.21.15.28:0/3497934462 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 0x56001b925810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa9a74640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b920a30 0x56001b992ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 -- 172.21.15.28:0/3497934462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b920a30 msgr2=0x56001b992ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b920a30 0x56001b992ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 -- 172.21.15.28:0/3497934462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 msgr2=0x56001b9252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa8a72640 1 -- 172.21.15.28:0/3497934462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56001b90d3f0 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.043+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aa8a72640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 0x56001b925810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009750 tx=0x7f8a940981b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45d5349649c7c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9401c040 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8a9409a150 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c011b60 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a9409b7b0 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 10 client.25558 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56001b99b8b0 con 0x56001b916170 2024-09-19T09:34:19.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c011b60 con 0x56001b916170 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8a940984c0 con 0x56001b916170 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 10 client.25558 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 10 client.25558 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 20 client.25558 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b99c0a0 0x56001b99e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 4 client.25558 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.044+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x56001b881be0 con 0x56001b99c0a0 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.045+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b99c0a0 0x56001b99e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:19.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.045+0000 7f8a8cff9640 10 client.25558 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:19.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.045+0000 7f8aa9273640 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b99c0a0 0x56001b99e4c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b9215f519c53d server_cookie=eebf695ef643d97e in_seq=0 out_seq=0 2024-09-19T09:34:19.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:19 smithi028 ceph-mon[30972]: pgmap v387: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 16 op/s 2024-09-19T09:34:19.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:19 smithi078 ceph-mon[35619]: pgmap v387: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 16 op/s 2024-09-19T09:34:19.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:19 smithi038 ceph-mon[35271]: pgmap v387: 97 pgs: 97 active+clean; 1.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 16 op/s 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.774+0000 7f8a8cff9640 1 -- 172.21.15.28:0/3497934462 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56001b881be0 con 0x56001b99c0a0 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.774+0000 7f8a8cff9640 10 client.25558 handle_command_reply: tid=0 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.774+0000 7f8aad8b0740 1 client.25558 shutdown 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 msgr2=0x56001b925810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 0x56001b925810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009750 tx=0x7f8a940981b0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 shutdown_connections 2024-09-19T09:34:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56001b920a30 0x56001b992ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56001b99c0a0 0x56001b99e4c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001b916170 0x56001b925810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 --2- 172.21.15.28:0/3497934462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56001b915850 0x56001b9252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8a8cff9640 0 client.25558 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 >> 172.21.15.28:0/3497934462 conn(0x56001b911de0 msgr2=0x56001b9106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 shutdown_connections 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 1 -- 172.21.15.28:0/3497934462 wait complete. 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.775+0000 7f8aad8b0740 20 client.25558 trim_cache size 0 max 16384 2024-09-19T09:34:19.778 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 10, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "ee32d0c4": { 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 10, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:34:19.779 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 140.3, 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:19.780 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 msgr2=0x7f8a900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 0x7f8a900a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0049d0 tx=0x7f8a9c009500 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 msgr2=0x7f8aa4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4100ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40697d0 tx=0x7f8a9400ee50 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 shutdown_connections 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8a900a3240 0x7f8a900a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa41082a0 0x7f8aa4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41036e0 0x7f8aa4100ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 --2- 172.21.15.28:0/1189397294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8aa4102d30 0x7f8aa4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 >> 172.21.15.28:0/1189397294 conn(0x7f8aa40fc7d0 msgr2=0x7f8aa4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:19.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.776+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 shutdown_connections 2024-09-19T09:34:19.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:19.777+0000 7f8aab4d7640 1 -- 172.21.15.28:0/1189397294 wait complete. 2024-09-19T09:34:20.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:20 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:34:20.384 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:20 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:34:20.420 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee32d0c4', quiesce took 6.2 seconds. Will release after: 14.150000000000002 2024-09-19T09:34:20.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:20 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:34:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 -- 172.21.15.28:0/3751436581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da81001d0 msgr2=0x7f5da81005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 --2- 172.21.15.28:0/3751436581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da81001d0 0x7f5da81005d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d9802a410 comp rx=0 tx=0).stop 2024-09-19T09:34:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 -- 172.21.15.28:0/3751436581 shutdown_connections 2024-09-19T09:34:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 --2- 172.21.15.28:0/3751436581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da8101d80 0x7f5da8108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 --2- 172.21.15.28:0/3751436581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81013c0 0x7f5da8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 --2- 172.21.15.28:0/3751436581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da81001d0 0x7f5da81005d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 -- 172.21.15.28:0/3751436581 >> 172.21.15.28:0/3751436581 conn(0x7f5da80fb970 msgr2=0x7f5da80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 -- 172.21.15.28:0/3751436581 shutdown_connections 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 -- 172.21.15.28:0/3751436581 wait complete. 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.216+0000 7f5dae027640 1 Processor -- start 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 -- start start 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 0x7f5da8109b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da81013c0 0x7f5da819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c710 con 0x7f5da8101d80 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c590 con 0x7f5da81013c0 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5dae027640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c890 con 0x7f5da81001d0 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39556/0 (socket says 172.21.15.28:39556) 2024-09-19T09:34:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 -- 172.21.15.28:0/839781461 learned_addr learned my addr 172.21.15.28:0/839781461 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da77fe640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 0x7f5da8109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da81013c0 0x7f5da819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 -- 172.21.15.28:0/839781461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 msgr2=0x7f5da8109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 0x7f5da8109b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 -- 172.21.15.28:0/839781461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da81013c0 msgr2=0x7f5da819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da81013c0 0x7f5da819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da7fff640 1 -- 172.21.15.28:0/839781461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da81a1cc0 con 0x7f5da8101d80 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.217+0000 7f5da77fe640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 0x7f5da8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5da7fff640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5da000cce0 tx=0x7f5da000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5da4ff9640 1 -- 172.21.15.28:0/839781461 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da000f450 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5da4ff9640 1 -- 172.21.15.28:0/839781461 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da000fad0 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da81a1f50 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5da4ff9640 1 -- 172.21.15.28:0/839781461 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0018c10 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.218+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81a2410 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.220+0000 7f5da4ff9640 1 -- 172.21.15.28:0/839781461 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5da000f5f0 con 0x7f5da8101d80 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.220+0000 7f5db0dc7740 1 Processor -- start 2024-09-19T09:34:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 -- start start 2024-09-19T09:34:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df4fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4fc370 0x55b4df4fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4df45a2f0 con 0x55b4df4f69a0 2024-09-19T09:34:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b4df402270 con 0x55b4df4fc370 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b4df4562b0 con 0x55b4df4f72e0 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39558/0 (socket says 172.21.15.28:39558) 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1333451322 learned_addr learned my addr 172.21.15.28:0/1333451322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da77fe640 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df4fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da4ff9640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 0x7f5d900a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1333451322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 msgr2=0x55b4df4fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df4fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1333451322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4fc370 msgr2=0x55b4df4fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4fc370 0x55b4df4fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1333451322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4df401340 con 0x55b4df4f69a0 2024-09-19T09:34:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.221+0000 7f5da7fff640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 0x7f5d900a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.222+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009d40 tx=0x7f5d9c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713ed633e3c84370 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.222+0000 7f5da4ff9640 1 -- 172.21.15.28:0/839781461 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5da00cfe70 con 0x7f5da8101d80 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.222+0000 7f5d8dffb640 1 -- 172.21.15.28:0/1333451322 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c004050 con 0x55b4df4f69a0 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.222+0000 7f5d8dffb640 1 -- 172.21.15.28:0/1333451322 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c017030 con 0x55b4df4f69a0 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.222+0000 7f5d8dffb640 1 -- 172.21.15.28:0/1333451322 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c010430 con 0x55b4df4f69a0 2024-09-19T09:34:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1333451322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 msgr2=0x55b4df4f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5da7fff640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 0x7f5d900a57b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5da00a4ba0 tx=0x7f5da006e230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009d40 tx=0x7f5d9c013830 comp rx=0 tx=0).stop 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1333451322 shutdown_connections 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4fc370 0x55b4df4fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df4fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1333451322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f69a0 0x55b4df4f6da0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1333451322 >> 172.21.15.28:0/1333451322 conn(0x55b4df4f2010 msgr2=0x55b4df4f2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1333451322 shutdown_connections 2024-09-19T09:34:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1333451322 wait complete. 2024-09-19T09:34:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 -- start start 2024-09-19T09:34:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.223+0000 7f5db0dc7740 1 Processor -- start 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df500b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4fc370 0x55b4df5080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4df4ff340 con 0x55b4df4f72e0 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b4df3e3290 con 0x55b4df4f69a0 2024-09-19T09:34:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b4df4ff4c0 con 0x55b4df4fc370 2024-09-19T09:34:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5db0dc7740 10 client.0 _ll_register_callbacks cb 0x55b4df414140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39572/0 (socket says 172.21.15.28:39572) 2024-09-19T09:34:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 -- 172.21.15.28:0/2748762652 learned_addr learned my addr 172.21.15.28:0/2748762652 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4fc370 0x55b4df5080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df500b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 -- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4fc370 msgr2=0x55b4df5080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4fc370 0x55b4df5080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 -- 172.21.15.28:0/2748762652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 msgr2=0x55b4df500b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df500b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 -- 172.21.15.28:0/2748762652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4df16d210 con 0x55b4df4f72e0 2024-09-19T09:34:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5da77fe640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5d980314e0 tx=0x7f5d98031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c906e487a2f2c45c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.224+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98035040 con 0x55b4df4f72e0 2024-09-19T09:34:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5d98004240 con 0x55b4df4f72e0 2024-09-19T09:34:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c017030 con 0x55b4df4f72e0 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d98004440 con 0x55b4df4f72e0 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 10 client.16011 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4df239620 con 0x55b4df4f72e0 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c017030 con 0x55b4df4f72e0 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5d98004ce0 con 0x55b4df4f72e0 2024-09-19T09:34:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 20 client.16011 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df50c790 0x55b4df50ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 4 client.16011 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.225+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4df467a20 con 0x55b4df50c790 2024-09-19T09:34:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.226+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df50c790 0x55b4df50ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.226+0000 7f5d86ffd640 10 client.16011 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.226+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df50c790 0x55b4df50ebc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb753999cc864d0 server_cookie=b64d83465c97ce11 in_seq=0 out_seq=0 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5d86ffd640 1 -- 172.21.15.28:0/2748762652 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4df467a20 con 0x55b4df50c790 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5d86ffd640 10 client.16011 handle_command_reply: tid=0 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 client.16011 shutdown 2024-09-19T09:34:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 msgr2=0x55b4df5010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5d980314e0 tx=0x7f5d98031a20 comp rx=0 tx=0).stop 2024-09-19T09:34:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 shutdown_connections 2024-09-19T09:34:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df50c790 0x55b4df50ebc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4fc370 0x55b4df5080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4f72e0 0x55b4df5010b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5d86ffd640 0 client.16011 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/2748762652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df500b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 >> 172.21.15.28:0/2748762652 conn(0x55b4df4f2010 msgr2=0x55b4df4f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 shutdown_connections 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 1 -- 172.21.15.28:0/2748762652 wait complete. 2024-09-19T09:34:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.227+0000 7f5db0dc7740 20 client.16011 trim_cache size 0 max 16384 2024-09-19T09:34:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 Processor -- start 2024-09-19T09:34:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 -- start start 2024-09-19T09:34:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df4fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df5080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81ab170 con 0x55b4df4fc370 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81accb0 con 0x55b4df4f69a0 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81ab840 con 0x55b4df4f72e0 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39582/0 (socket says 172.21.15.28:39582) 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1738358322 learned_addr learned my addr 172.21.15.28:0/1738358322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.242+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df4fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da77fe640 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df5080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1738358322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 msgr2=0x55b4df5080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df5080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1738358322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 msgr2=0x55b4df4fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df4fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1738358322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4df4ec2f0 con 0x55b4df4fc370 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009d10 tx=0x7f5d9c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4a36ca3b7edd5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5d8d7fa640 1 -- 172.21.15.28:0/1738358322 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c013180 con 0x55b4df4fc370 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5d8d7fa640 1 -- 172.21.15.28:0/1738358322 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c017070 con 0x55b4df4fc370 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.243+0000 7f5d8d7fa640 1 -- 172.21.15.28:0/1738358322 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0051a0 con 0x55b4df4fc370 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1738358322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 msgr2=0x55b4df4f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009d10 tx=0x7f5d9c01e040 comp rx=0 tx=0).stop 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1738358322 shutdown_connections 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df4f10b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df5080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1738358322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df4fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1738358322 >> 172.21.15.28:0/1738358322 conn(0x55b4df4f2010 msgr2=0x55b4df4f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1738358322 shutdown_connections 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1738358322 wait complete. 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 -- start start 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.244+0000 7f5db0dc7740 1 Processor -- start 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df505bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df506110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81ac5e0 con 0x55b4df4fc370 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81abf10 con 0x55b4df4f69a0 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81a9150 con 0x55b4df4f72e0 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5db0dc7740 10 client.0 _ll_register_callbacks cb 0x7f5d98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39596/0 (socket says 172.21.15.28:39596) 2024-09-19T09:34:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1391119915 learned_addr learned my addr 172.21.15.28:0/1391119915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da77fe640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df506110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df505bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 msgr2=0x55b4df506110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df506110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1391119915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 msgr2=0x55b4df505bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df505bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 -- 172.21.15.28:0/1391119915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4df507730 con 0x55b4df4fc370 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5da6ffd640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b4df5051c0 tx=0x7f5d9c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6038fa5afc2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.245+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003f20 con 0x55b4df4fc370 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5d9c03e070 con 0x55b4df4fc370 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c017070 con 0x55b4df4fc370 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d9c0040c0 con 0x55b4df4fc370 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 10 client.16017 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4df5778a0 con 0x55b4df4fc370 2024-09-19T09:34:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c017070 con 0x55b4df4fc370 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5d9c041050 con 0x55b4df4fc370 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 10 client.16017 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 20 client.16017 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df577fd0 0x55b4df57a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 4 client.16017 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:34:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4df461d70 con 0x55b4df577fd0 2024-09-19T09:34:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.246+0000 7f5da77fe640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df577fd0 0x55b4df57a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.247+0000 7f5da77fe640 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df577fd0 0x55b4df57a3f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ed3f37e7f14aba server_cookie=efeb9396e6a2b6e2 in_seq=0 out_seq=0 2024-09-19T09:34:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.247+0000 7f5d86ffd640 10 client.16017 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.276+0000 7f5d86ffd640 1 -- 172.21.15.28:0/1391119915 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4df461d70 con 0x55b4df577fd0 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.276+0000 7f5d86ffd640 10 client.16017 handle_command_reply: tid=0 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.276+0000 7f5db0dc7740 1 client.16017 shutdown 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 msgr2=0x55b4df507120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55b4df5051c0 tx=0x7f5d9c015040 comp rx=0 tx=0).stop 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 shutdown_connections 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b4df577fd0 0x55b4df57a3f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4df4fc370 0x55b4df507120 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4df4f72e0 0x55b4df506110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5d86ffd640 0 client.16017 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 --2- 172.21.15.28:0/1391119915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b4df4f69a0 0x55b4df505bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 >> 172.21.15.28:0/1391119915 conn(0x55b4df4f2010 msgr2=0x55b4df4f2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 shutdown_connections 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 1 -- 172.21.15.28:0/1391119915 wait complete. 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5db0dc7740 20 client.16017 trim_cache size 0 max 16384 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:34:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:34:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:34:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:34:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 msgr2=0x7f5d900a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 0x7f5d900a57b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5da00a4ba0 tx=0x7f5da006e230 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 msgr2=0x7f5da81a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5da000cce0 tx=0x7f5da000a620 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 shutdown_connections 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5d900a32f0 0x7f5d900a57b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.277+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da81013c0 0x7f5da819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.278+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5da8101d80 0x7f5da81a14d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.278+0000 7f5dae027640 1 --2- 172.21.15.28:0/839781461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da81001d0 0x7f5da8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.278+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 >> 172.21.15.28:0/839781461 conn(0x7f5da80fb970 msgr2=0x7f5da8104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.278+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 shutdown_connections 2024-09-19T09:34:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:21.278+0000 7f5dae027640 1 -- 172.21.15.28:0/839781461 wait complete. 2024-09-19T09:34:21.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:21 smithi078 ceph-mon[35619]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 152 KiB/s wr, 27 op/s 2024-09-19T09:34:21.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:21 smithi038 ceph-mon[35271]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 152 KiB/s wr, 27 op/s 2024-09-19T09:34:21.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:21 smithi028 ceph-mon[30972]: pgmap v388: 97 pgs: 97 active+clean; 3.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 152 KiB/s wr, 27 op/s 2024-09-19T09:34:23.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:23 smithi078 ceph-mon[35619]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 163 KiB/s wr, 28 op/s 2024-09-19T09:34:23.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:23 smithi038 ceph-mon[35271]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 163 KiB/s wr, 28 op/s 2024-09-19T09:34:23.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:23 smithi028 ceph-mon[30972]: pgmap v389: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 163 KiB/s wr, 28 op/s 2024-09-19T09:34:25.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:25 smithi078 ceph-mon[35619]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 195 KiB/s wr, 45 op/s 2024-09-19T09:34:25.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:25 smithi038 ceph-mon[35271]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 195 KiB/s wr, 45 op/s 2024-09-19T09:34:25.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:25 smithi028 ceph-mon[30972]: pgmap v390: 97 pgs: 97 active+clean; 3.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 195 KiB/s wr, 45 op/s 2024-09-19T09:34:27.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:27 smithi078 ceph-mon[35619]: pgmap v391: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-19T09:34:27.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:27 smithi038 ceph-mon[35271]: pgmap v391: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-19T09:34:27.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:27 smithi028 ceph-mon[30972]: pgmap v391: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-19T09:34:29.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:29 smithi078 ceph-mon[35619]: pgmap v392: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 56 op/s 2024-09-19T09:34:29.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:29 smithi038 ceph-mon[35271]: pgmap v392: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 56 op/s 2024-09-19T09:34:29.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:29 smithi028 ceph-mon[30972]: pgmap v392: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 56 op/s 2024-09-19T09:34:31.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:31 smithi078 ceph-mon[35619]: pgmap v393: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 224 KiB/s wr, 55 op/s 2024-09-19T09:34:31.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:31 smithi038 ceph-mon[35271]: pgmap v393: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 224 KiB/s wr, 55 op/s 2024-09-19T09:34:31.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:31 smithi028 ceph-mon[30972]: pgmap v393: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 224 KiB/s wr, 55 op/s 2024-09-19T09:34:33.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:33 smithi078 ceph-mon[35619]: pgmap v394: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T09:34:33.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:33 smithi038 ceph-mon[35271]: pgmap v394: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T09:34:33.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:33 smithi028 ceph-mon[30972]: pgmap v394: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 -- 172.21.15.28:0/1357307797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8102d50 msgr2=0x7f7af81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 --2- 172.21.15.28:0/1357307797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8102d50 0x7f7af81031d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02a410 comp rx=0 tx=0).stop 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 -- 172.21.15.28:0/1357307797 shutdown_connections 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 --2- 172.21.15.28:0/1357307797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8102d50 0x7f7af81031d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 --2- 172.21.15.28:0/1357307797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 --2- 172.21.15.28:0/1357307797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8101b60 0x7f7af8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 -- 172.21.15.28:0/1357307797 >> 172.21.15.28:0/1357307797 conn(0x7f7af80fd2e0 msgr2=0x7f7af80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 -- 172.21.15.28:0/1357307797 shutdown_connections 2024-09-19T09:34:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.377+0000 7f7afea65640 1 -- 172.21.15.28:0/1357307797 wait complete. 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 Processor -- start 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 -- start start 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 0x7f7af81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d7c0 con 0x7f7af8103710 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d640 con 0x7f7af8101b60 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afea65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d940 con 0x7f7af8102d50 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afc801640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afd002640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afd002640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38826/0 (socket says 172.21.15.28:38826) 2024-09-19T09:34:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afc801640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:35590/0 (socket says 172.21.15.28:35590) 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.378+0000 7f7afd002640 1 -- 172.21.15.28:0/6871492 learned_addr learned my addr 172.21.15.28:0/6871492 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7af7fff640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 0x7f7af81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 -- 172.21.15.28:0/6871492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 msgr2=0x7f7af81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 0x7f7af81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 -- 172.21.15.28:0/6871492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 msgr2=0x7f7af8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 -- 172.21.15.28:0/6871492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819faa0 con 0x7f7af8103710 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7af7fff640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 0x7f7af81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afc801640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afd002640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7af000eb10 tx=0x7f7af000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7af5ffb640 1 -- 172.21.15.28:0/6871492 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0004b20 con 0x7f7af8103710 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7af5ffb640 1 -- 172.21.15.28:0/6871492 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af0004cc0 con 0x7f7af8103710 2024-09-19T09:34:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819fd90 con 0x7f7af8103710 2024-09-19T09:34:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.379+0000 7f7af5ffb640 1 -- 172.21.15.28:0/6871492 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0010760 con 0x7f7af8103710 2024-09-19T09:34:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.380+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af81a0200 con 0x7f7af8103710 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7af5ffb640 1 -- 172.21.15.28:0/6871492 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7af0005020 con 0x7f7af8103710 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 Processor -- start 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 -- start start 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a41370 0x55e710a43830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e71099f2f0 con 0x55e710a3b9a0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e710947270 con 0x55e710a3c2e0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7b01807740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e71099b2b0 con 0x55e710a41370 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7af5ffb640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 0x7f7ae00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38838/0 (socket says 172.21.15.28:38838) 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7af7fff640 1 -- 172.21.15.28:0/2458581332 learned_addr learned my addr 172.21.15.28:0/2458581332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.383+0000 7f7afc801640 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7afd002640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 0x7f7ae00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a41370 0x55e710a43830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af5ffb640 1 -- 172.21.15.28:0/6871492 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7af00cff40 con 0x7f7af8103710 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 -- 172.21.15.28:0/2458581332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a41370 msgr2=0x55e710a43830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a41370 0x55e710a43830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 -- 172.21.15.28:0/2458581332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 msgr2=0x55e710a40da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a40da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 -- 172.21.15.28:0/2458581332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e710946340 con 0x55e710a3b9a0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7afd002640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 0x7f7ae00a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7af00989f0 tx=0x7f7af0091a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7af7fff640 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7aec005ab0 tx=0x7f7aec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f38746209e184a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7adeffd640 1 -- 172.21.15.28:0/2458581332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec035050 con 0x55e710a3b9a0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7adeffd640 1 -- 172.21.15.28:0/2458581332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0319c0 con 0x55e710a3b9a0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.384+0000 7f7adeffd640 1 -- 172.21.15.28:0/2458581332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004920 con 0x55e710a3b9a0 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- 172.21.15.28:0/2458581332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 msgr2=0x55e710a3bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7aec005ab0 tx=0x7f7aec005b50 comp rx=0 tx=0).stop 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- 172.21.15.28:0/2458581332 shutdown_connections 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a41370 0x55e710a43830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 --2- 172.21.15.28:0/2458581332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a3bda0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- 172.21.15.28:0/2458581332 >> 172.21.15.28:0/2458581332 conn(0x55e710a37010 msgr2=0x55e710a37410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- 172.21.15.28:0/2458581332 shutdown_connections 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- 172.21.15.28:0/2458581332 wait complete. 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.385+0000 7f7b01807740 1 -- start start 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 Processor -- start 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3b9a0 0x55e710a49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a4a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e710a442e0 con 0x55e710a41370 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e710928290 con 0x55e710a3c2e0 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e710a44460 con 0x55e710a3b9a0 2024-09-19T09:34:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7b01807740 10 client.0 _ll_register_callbacks cb 0x55e710959140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38850/0 (socket says 172.21.15.28:38850) 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7af7fff640 1 -- 172.21.15.28:0/4008767393 learned_addr learned my addr 172.21.15.28:0/4008767393 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7af7fff640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3b9a0 0x55e710a49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.386+0000 7f7afc801640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a4a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 -- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3b9a0 msgr2=0x55e710a49fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3b9a0 0x55e710a49fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 -- 172.21.15.28:0/4008767393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 msgr2=0x55e710a4a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a4a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 -- 172.21.15.28:0/4008767393 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7106b2210 con 0x55e710a41370 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7af7fff640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7aec031610 tx=0x7f7aec03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317a63efee89009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec035070 con 0x55e710a41370 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7aec004c50 con 0x55e710a41370 2024-09-19T09:34:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aec0319c0 con 0x55e710a41370 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7aec004e50 con 0x55e710a41370 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7b01807740 10 client.16029 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e71077e620 con 0x55e710a41370 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.387+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0319c0 con 0x55e710a41370 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7aec03a480 con 0x55e710a41370 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 20 client.16029 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a55820 0x55e710a57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 4 client.16029 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7109aca20 con 0x55e710a55820 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7afc801640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a55820 0x55e710a57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7afc801640 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a55820 0x55e710a57c10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940eac4cce6b98ac server_cookie=1005bb27106d605c in_seq=0 out_seq=0 2024-09-19T09:34:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.388+0000 7f7ad7fff640 10 client.16029 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7ad7fff640 1 -- 172.21.15.28:0/4008767393 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7109aca20 con 0x55e710a55820 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7ad7fff640 10 client.16029 handle_command_reply: tid=0 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 client.16029 shutdown 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 msgr2=0x55e710a51520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7aec031610 tx=0x7f7aec03f040 comp rx=0 tx=0).stop 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 shutdown_connections 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a4a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a55820 0x55e710a57c10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a41370 0x55e710a51520 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 --2- 172.21.15.28:0/4008767393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3b9a0 0x55e710a49fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7ad7fff640 0 client.16029 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.389+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 >> 172.21.15.28:0/4008767393 conn(0x55e710a37010 msgr2=0x55e710a38820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.390+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 shutdown_connections 2024-09-19T09:34:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.390+0000 7f7b01807740 1 -- 172.21.15.28:0/4008767393 wait complete. 2024-09-19T09:34:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.390+0000 7f7b01807740 20 client.16029 trim_cache size 0 max 16384 2024-09-19T09:34:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.404+0000 7f7b01807740 1 Processor -- start 2024-09-19T09:34:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.404+0000 7f7b01807740 1 -- start start 2024-09-19T09:34:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a37010 0x55e710a37ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a38400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a6b90 con 0x55e710a3b9a0 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a86d0 con 0x55e710a3c2e0 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7b01807740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a7260 con 0x55e710a37010 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38854/0 (socket says 172.21.15.28:38854) 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 -- 172.21.15.28:0/3372165870 learned_addr learned my addr 172.21.15.28:0/3372165870 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7af7fff640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a38400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7af7fff640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a37010 0x55e710a37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 -- 172.21.15.28:0/3372165870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a37010 msgr2=0x55e710a37ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a37010 0x55e710a37ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 -- 172.21.15.28:0/3372165870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 msgr2=0x55e710a38400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a38400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 -- 172.21.15.28:0/3372165870 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e710a58260 con 0x55e710a3b9a0 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7af7fff640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a38400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7afc801640 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800baf0 tx=0x7f7ae800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62814f5324fa1447 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7ade7fc640 1 -- 172.21.15.28:0/3372165870 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8005220 con 0x55e710a3b9a0 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7ade7fc640 1 -- 172.21.15.28:0/3372165870 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae8004c80 con 0x55e710a3b9a0 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.405+0000 7f7ade7fc640 1 -- 172.21.15.28:0/3372165870 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8003aa0 con 0x55e710a3b9a0 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- 172.21.15.28:0/3372165870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 msgr2=0x55e710a51440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800baf0 tx=0x7f7ae800b390 comp rx=0 tx=0).stop 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- 172.21.15.28:0/3372165870 shutdown_connections 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a3c2e0 0x55e710a38400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a51440 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 --2- 172.21.15.28:0/3372165870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a37010 0x55e710a37ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- 172.21.15.28:0/3372165870 >> 172.21.15.28:0/3372165870 conn(0x55e710a40e90 msgr2=0x55e710a51f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- 172.21.15.28:0/3372165870 shutdown_connections 2024-09-19T09:34:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- 172.21.15.28:0/3372165870 wait complete. 2024-09-19T09:34:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.406+0000 7f7b01807740 1 -- start start 2024-09-19T09:34:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 Processor -- start 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a37010 0x55e710a4e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3c2e0 0x55e710a4fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a8000 con 0x55e710a3b9a0 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a7930 con 0x55e710a37010 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81a4b70 con 0x55e710a3c2e0 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7b01807740 10 client.0 _ll_register_callbacks cb 0x7f7ae800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38858/0 (socket says 172.21.15.28:38858) 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 -- 172.21.15.28:0/1447453900 learned_addr learned my addr 172.21.15.28:0/1447453900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7af7fff640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a37010 0x55e710a4e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7af7fff640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3c2e0 0x55e710a4fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 -- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3c2e0 msgr2=0x55e710a4fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3c2e0 0x55e710a4fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 -- 172.21.15.28:0/1447453900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a37010 msgr2=0x55e710a4e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a37010 0x55e710a4e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.407+0000 7f7afc801640 1 -- 172.21.15.28:0/1447453900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e710a322d0 con 0x55e710a3b9a0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7afc801640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011fd0 tx=0x7f7ae800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb715ca3b1308776 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8034060 con 0x55e710a3b9a0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7ae8005020 con 0x55e710a3b9a0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae8004c80 con 0x55e710a3b9a0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ae8005220 con 0x55e710a3b9a0 2024-09-19T09:34:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 10 client.16035 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e710a51590 con 0x55e710a3b9a0 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae8004c80 con 0x55e710a3b9a0 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ae8026540 con 0x55e710a3b9a0 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 20 client.16035 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a41370 0x55e710ac2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 4 client.16035 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.408+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7109a6d70 con 0x55e710a41370 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.409+0000 7f7af7fff640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a41370 0x55e710ac2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.409+0000 7f7af7fff640 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a41370 0x55e710ac2a60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb4a70b73b4907d server_cookie=53f1f8ab1632bf22 in_seq=0 out_seq=0 2024-09-19T09:34:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.409+0000 7f7ad7fff640 10 client.16035 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7ad7fff640 1 -- 172.21.15.28:0/1447453900 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7109a6d70 con 0x55e710a41370 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7ad7fff640 10 client.16035 handle_command_reply: tid=0 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 client.16035 shutdown 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 msgr2=0x55e710a4ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011fd0 tx=0x7f7ae800bc80 comp rx=0 tx=0).stop 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 shutdown_connections 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e710a41370 0x55e710ac2a60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e710a3c2e0 0x55e710a4fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e710a3b9a0 0x55e710a4ee80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 --2- 172.21.15.28:0/1447453900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e710a37010 0x55e710a4e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7ad7fff640 0 client.16035 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 >> 172.21.15.28:0/1447453900 conn(0x55e710a40e90 msgr2=0x55e710a359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 shutdown_connections 2024-09-19T09:34:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 1 -- 172.21.15.28:0/1447453900 wait complete. 2024-09-19T09:34:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.422+0000 7f7b01807740 20 client.16035 trim_cache size 0 max 16384 2024-09-19T09:34:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:34:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:34:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:34:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:34:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 msgr2=0x7f7ae00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 0x7f7ae00a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7af00989f0 tx=0x7f7af0091a70 comp rx=0 tx=0).stop 2024-09-19T09:34:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 msgr2=0x7f7af819f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7af000eb10 tx=0x7f7af000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 shutdown_connections 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ae00a3240 0x7f7ae00a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7af8103710 0x7f7af819f430 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af8102d50 0x7f7af81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 --2- 172.21.15.28:0/6871492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af8101b60 0x7f7af8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 >> 172.21.15.28:0/6871492 conn(0x7f7af80fd2e0 msgr2=0x7f7af8106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 shutdown_connections 2024-09-19T09:34:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:34.423+0000 7f7afea65640 1 -- 172.21.15.28:0/6871492 wait complete. 2024-09-19T09:34:34.571 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:34:34.830 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:35.215 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:35 smithi028 ceph-mon[30972]: pgmap v395: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 40 op/s 2024-09-19T09:34:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 -- 172.21.15.28:0/893698732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa0010e480 msgr2=0x7faa00110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 --2- 172.21.15.28:0/893698732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa0010e480 0x7faa00110870 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009900 tx=0x7fa9f802a410 comp rx=0 tx=0).stop 2024-09-19T09:34:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 -- 172.21.15.28:0/893698732 shutdown_connections 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 --2- 172.21.15.28:0/893698732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 0x7faa0010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 --2- 172.21.15.28:0/893698732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa0010e480 0x7faa00110870 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 --2- 172.21.15.28:0/893698732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa001008d0 0x7faa00100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 -- 172.21.15.28:0/893698732 >> 172.21.15.28:0/893698732 conn(0x7faa000fc7a0 msgr2=0x7faa000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 -- 172.21.15.28:0/893698732 shutdown_connections 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.295+0000 7faa07ba6640 1 -- 172.21.15.28:0/893698732 wait complete. 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 Processor -- start 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 -- start start 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 0x7faa0019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa0010e480 0x7faa0019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa00114170 con 0x7faa001008d0 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa00113ff0 con 0x7faa00101280 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa07ba6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa001142f0 con 0x7faa0010e480 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38876/0 (socket says 172.21.15.28:38876) 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 -- 172.21.15.28:0/1289401818 learned_addr learned my addr 172.21.15.28:0/1289401818 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05141640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 0x7faa0019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa06143640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa0010e480 0x7faa0019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 -- 172.21.15.28:0/1289401818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa0010e480 msgr2=0x7faa0019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa0010e480 0x7faa0019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 -- 172.21.15.28:0/1289401818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 msgr2=0x7faa0019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.296+0000 7faa05942640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 0x7faa0019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7faa05942640 1 -- 172.21.15.28:0/1289401818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0019f620 con 0x7faa001008d0 2024-09-19T09:34:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7faa05942640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000ea80 tx=0x7fa9f000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000ce40 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0005020 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0019f910 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0011760 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.297+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a71f0 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.298+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa00100cb0 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.299+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa9f00051d0 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.300+0000 7fa9feffd640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 0x7fa9e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.300+0000 7faa05141640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 0x7fa9e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.300+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f00cfea0 con 0x7faa001008d0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.301+0000 7faa05141640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 0x7fa9e80a5780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faa00102370 tx=0x7fa9f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.304+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9f0016030 con 0x7faa001008d0 2024-09-19T09:34:35.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:35 smithi078 ceph-mon[35619]: pgmap v395: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 40 op/s 2024-09-19T09:34:35.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:35 smithi038 ceph-mon[35271]: pgmap v395: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 40 op/s 2024-09-19T09:34:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.519+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa0019b9d0 con 0x7faa001008d0 2024-09-19T09:34:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.519+0000 7fa9feffd640 1 -- 172.21.15.28:0/1289401818 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11951 (secure 0 0 0) 0x7fa9f009c5c0 con 0x7faa001008d0 2024-09-19T09:34:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:34:35.522 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"btime":"2024-09-19T09:32:28:608681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:32:27.697965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:34:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 msgr2=0x7fa9e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 0x7fa9e80a5780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faa00102370 tx=0x7fa9f400b390 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 msgr2=0x7faa0019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000ea80 tx=0x7fa9f000ef40 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 shutdown_connections 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9e80a32c0 0x7fa9e80a5780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa00101280 0x7faa0019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa0010e480 0x7faa0019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 --2- 172.21.15.28:0/1289401818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa001008d0 0x7faa0019a670 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:35.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 >> 172.21.15.28:0/1289401818 conn(0x7faa000fc7a0 msgr2=0x7faa000feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:35.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 shutdown_connections 2024-09-19T09:34:35.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:35.522+0000 7faa07ba6640 1 -- 172.21.15.28:0/1289401818 wait complete. 2024-09-19T09:34:36.186 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:36 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1289401818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:36.218 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id ee32d0c4 --release --await' 2024-09-19T09:34:36.218 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id ee32d0c4 --release --await 2024-09-19T09:34:36.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:36 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1289401818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:36.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:36 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1289401818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:36.478 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:36.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 -- 172.21.15.28:0/508986788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 msgr2=0x7fdea0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 --2- 172.21.15.28:0/508986788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea0106e60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde90009930 tx=0x7fde9002a420 comp rx=0 tx=0).stop 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 -- 172.21.15.28:0/508986788 shutdown_connections 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 --2- 172.21.15.28:0/508986788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 0x7fdea010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 --2- 172.21.15.28:0/508986788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 0x7fdea0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 --2- 172.21.15.28:0/508986788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea0106e60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 -- 172.21.15.28:0/508986788 >> 172.21.15.28:0/508986788 conn(0x7fdea00fc7a0 msgr2=0x7fdea00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 -- 172.21.15.28:0/508986788 shutdown_connections 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.960+0000 7fdea698c640 1 -- 172.21.15.28:0/508986788 wait complete. 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 Processor -- start 2024-09-19T09:34:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 -- start start 2024-09-19T09:34:36.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 0x7fdea0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 0x7fdea0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0110e80 con 0x7fdea0106a80 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0110d00 con 0x7fdea0101420 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea698c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0111000 con 0x7fdea0100a80 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea4f29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea4f29640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38888/0 (socket says 172.21.15.28:38888) 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fdea4f29640 1 -- 172.21.15.28:0/920292156 learned_addr learned my addr 172.21.15.28:0/920292156 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 0x7fdea0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.961+0000 7fde9ffff640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 0x7fdea0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 -- 172.21.15.28:0/920292156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 msgr2=0x7fdea0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 0x7fdea0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 -- 172.21.15.28:0/920292156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 msgr2=0x7fdea0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 0x7fdea0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 -- 172.21.15.28:0/920292156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea019d3e0 con 0x7fdea0106a80 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea4f29640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fde9800eaa0 tx=0x7fde9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fde9d7fa640 1 -- 172.21.15.28:0/920292156 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9800cdb0 con 0x7fdea0106a80 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea019d6d0 con 0x7fdea0106a80 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fde9d7fa640 1 -- 172.21.15.28:0/920292156 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde98005020 con 0x7fdea0106a80 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fde9d7fa640 1 -- 172.21.15.28:0/920292156 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde980116b0 con 0x7fdea0106a80 2024-09-19T09:34:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.962+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea01a4fb0 con 0x7fdea0106a80 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9d7fa640 1 -- 172.21.15.28:0/920292156 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fde98005560 con 0x7fdea0106a80 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 Processor -- start 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 -- start start 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 0x56032b59a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b5a5240 0x56032b5a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56032b4ff620 con 0x56032b59a980 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56032b4a6e50 con 0x56032b59a060 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea8d65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56032b4fb5e0 con 0x56032b5a5240 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9d7fa640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 0x7fde880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 0x56032b59a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38892/0 (socket says 172.21.15.28:38892) 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fde9ffff640 1 -- 172.21.15.28:0/3973253011 learned_addr learned my addr 172.21.15.28:0/3973253011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.965+0000 7fdea4f29640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 0x7fde880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b5a5240 0x56032b5a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9d7fa640 1 -- 172.21.15.28:0/920292156 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fde980cfa90 con 0x7fdea0106a80 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 -- 172.21.15.28:0/3973253011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b5a5240 msgr2=0x56032b5a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b5a5240 0x56032b5a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 -- 172.21.15.28:0/3973253011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 msgr2=0x56032b59a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 0x56032b59a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 -- 172.21.15.28:0/3973253011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032b497d40 con 0x56032b59a980 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 0x56032b59a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fdea4f29640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 0x7fde880a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde980a47c0 tx=0x7fde98066b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde9ffff640 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde9002a940 tx=0x7fde90005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbabdc66b1793878 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde867fc640 1 -- 172.21.15.28:0/3973253011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90040070 con 0x56032b59a980 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde867fc640 1 -- 172.21.15.28:0/3973253011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90031930 con 0x56032b59a980 2024-09-19T09:34:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.966+0000 7fde867fc640 1 -- 172.21.15.28:0/3973253011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90035040 con 0x56032b59a980 2024-09-19T09:34:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- 172.21.15.28:0/3973253011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 msgr2=0x56032b5a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde9002a940 tx=0x7fde90005c00 comp rx=0 tx=0).stop 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- 172.21.15.28:0/3973253011 shutdown_connections 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b5a5240 0x56032b5a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a980 0x56032b5a4d00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 --2- 172.21.15.28:0/3973253011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a060 0x56032b59a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- 172.21.15.28:0/3973253011 >> 172.21.15.28:0/3973253011 conn(0x56032b5965f0 msgr2=0x56032b5969d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- 172.21.15.28:0/3973253011 shutdown_connections 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- 172.21.15.28:0/3973253011 wait complete. 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.967+0000 7fdea8d65740 1 -- start start 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 Processor -- start 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b5a2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b5a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56032b5ac220 con 0x56032b59a060 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56032b487f10 con 0x56032b5a5240 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56032b5ac3a0 con 0x56032b59a980 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fdea8d65740 10 client.0 _ll_register_callbacks cb 0x56032b4abdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38900/0 (socket says 172.21.15.28:38900) 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1061282819 learned_addr learned my addr 172.21.15.28:0/1061282819 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9ffff640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b5a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b5a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 msgr2=0x56032b5a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b5a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1061282819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 msgr2=0x56032b5a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b5a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.968+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1061282819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032b20e580 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde94003620 tx=0x7fde94014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6d4431177eded9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940129a0 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde90031930 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fde94012b40 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde94012d40 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fdea8d65740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56032b2d8960 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.969+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90031930 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fde9400f3f0 con 0x56032b59a060 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 10 client.16053 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 20 client.16053 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b5b90c0 0x56032b5bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 4 client.16053 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56032b50cb90 con 0x56032b5b90c0 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fde9ffff640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b5b90c0 0x56032b5bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fde9ffff640 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b5b90c0 0x56032b5bb4d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff90f69c0e1d3706 server_cookie=5e48f6711734b89f in_seq=0 out_seq=0 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.970+0000 7fde7f7fe640 10 client.16053 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fde7f7fe640 1 -- 172.21.15.28:0/1061282819 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56032b50cb90 con 0x56032b5b90c0 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fde7f7fe640 10 client.16053 handle_command_reply: tid=0 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 client.16053 shutdown 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 msgr2=0x56032b5a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde94003620 tx=0x7fde94014040 comp rx=0 tx=0).stop 2024-09-19T09:34:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 shutdown_connections 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b5a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b5b90c0 0x56032b5bb4d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b5a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1061282819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b5a1fd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 >> 172.21.15.28:0/1061282819 conn(0x56032b5965f0 msgr2=0x56032b595750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.971+0000 7fde7f7fe640 0 client.16053 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.972+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 shutdown_connections 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.972+0000 7fdea8d65740 1 -- 172.21.15.28:0/1061282819 wait complete. 2024-09-19T09:34:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.972+0000 7fdea8d65740 20 client.16053 trim_cache size 0 max 16384 2024-09-19T09:34:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 Processor -- start 2024-09-19T09:34:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 -- start start 2024-09-19T09:34:36.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a060 0x56032b5c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a980 0x56032b5c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ab290 con 0x56032b5a5240 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01acdd0 con 0x56032b59a980 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fdea8d65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ab960 con 0x56032b59a060 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38908/0 (socket says 172.21.15.28:38908) 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1992420370 learned_addr learned my addr 172.21.15.28:0/1992420370 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9ffff640 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a980 0x56032b5c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a060 0x56032b5c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1992420370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a060 msgr2=0x56032b5c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a060 0x56032b5c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1992420370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a980 msgr2=0x56032b5c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a980 0x56032b5c12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 -- 172.21.15.28:0/1992420370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032b5971a0 con 0x56032b5a5240 2024-09-19T09:34:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.989+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde94004010 tx=0x7fde94004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b1ec7d257664fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fde85ffb640 1 -- 172.21.15.28:0/1992420370 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940127a0 con 0x56032b5a5240 2024-09-19T09:34:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fde85ffb640 1 -- 172.21.15.28:0/1992420370 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde94012940 con 0x56032b5a5240 2024-09-19T09:34:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fde85ffb640 1 -- 172.21.15.28:0/1992420370 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94024a70 con 0x56032b5a5240 2024-09-19T09:34:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 -- 172.21.15.28:0/1992420370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 msgr2=0x56032b59f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde94004010 tx=0x7fde94004120 comp rx=0 tx=0).stop 2024-09-19T09:34:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 -- 172.21.15.28:0/1992420370 shutdown_connections 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b59a980 0x56032b5c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b5a5240 0x56032b59f250 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 --2- 172.21.15.28:0/1992420370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a060 0x56032b5c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 -- 172.21.15.28:0/1992420370 >> 172.21.15.28:0/1992420370 conn(0x56032b5965f0 msgr2=0x56032b5a4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 -- 172.21.15.28:0/1992420370 shutdown_connections 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.990+0000 7fdea8d65740 1 -- 172.21.15.28:0/1992420370 wait complete. 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 -- start start 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 Processor -- start 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b59f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b59fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ac700 con 0x56032b59a060 2024-09-19T09:34:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ac030 con 0x56032b5a5240 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01a9270 con 0x56032b59a980 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fdea8d65740 10 client.0 _ll_register_callbacks cb 0x56032b4abdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38914/0 (socket says 172.21.15.28:38914) 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.991+0000 7fde9f7fe640 1 -- 172.21.15.28:0/93035015 learned_addr learned my addr 172.21.15.28:0/93035015 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9ffff640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b59f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 -- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 msgr2=0x56032b59f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b59f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 -- 172.21.15.28:0/93035015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 msgr2=0x56032b59fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b59fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 -- 172.21.15.28:0/93035015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032b59a7b0 con 0x56032b59a060 2024-09-19T09:34:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde9f7fe640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fde94012d00 tx=0x7fde9400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb683c7683292d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94035020 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fde94005050 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde94012940 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde940127a0 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fdea8d65740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56032b624130 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde94012940 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.992+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fde94004580 con 0x56032b59a060 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 10 client.16059 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 20 client.16059 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:36.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b6245b0 0x56032b6269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 4 client.16059 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee32d0c4", "release": true, "await": true}] 2024-09-19T09:34:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee32d0c4", "release": true, "await": true}) -- 0x56032b5070a0 con 0x56032b6245b0 2024-09-19T09:34:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fde9ffff640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b6245b0 0x56032b6269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fde9ffff640 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b6245b0 0x56032b6269a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967f5aa4ac35089d server_cookie=753d065d9de77007 in_seq=0 out_seq=0 2024-09-19T09:34:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.993+0000 7fde7f7fe640 10 client.16059 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fde7f7fe640 1 -- 172.21.15.28:0/93035015 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56032b5070a0 con 0x56032b6245b0 2024-09-19T09:34:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fde7f7fe640 10 client.16059 handle_command_reply: tid=0 2024-09-19T09:34:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 client.16059 shutdown 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 msgr2=0x56032b59f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fde94012d00 tx=0x7fde9400b390 comp rx=0 tx=0).stop 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 shutdown_connections 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56032b5a5240 0x56032b59fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56032b6245b0 0x56032b6269a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56032b59a980 0x56032b59f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fde7f7fe640 0 client.16059 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 --2- 172.21.15.28:0/93035015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56032b59a060 0x56032b59f1e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 >> 172.21.15.28:0/93035015 conn(0x56032b5965f0 msgr2=0x56032b59bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 shutdown_connections 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 1 -- 172.21.15.28:0/93035015 wait complete. 2024-09-19T09:34:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:36.999+0000 7fdea8d65740 20 client.16059 trim_cache size 0 max 16384 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 27, 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 12, 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "ee32d0c4": { 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 12, 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:37.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 140.3, 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:34:37.004 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 msgr2=0x7fde880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 0x7fde880a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde980a47c0 tx=0x7fde98066b70 comp rx=0 tx=0).stop 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 msgr2=0x7fdea019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fde9800eaa0 tx=0x7fde9800ef60 comp rx=0 tx=0).stop 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 shutdown_connections 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fde880a3240 0x7fde880a5700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdea0101420 0x7fdea0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdea0106a80 0x7fdea019cd00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 --2- 172.21.15.28:0/920292156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdea0100a80 0x7fdea0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 >> 172.21.15.28:0/920292156 conn(0x7fdea00fc7a0 msgr2=0x7fdea006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 shutdown_connections 2024-09-19T09:34:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:37.000+0000 7fdea698c640 1 -- 172.21.15.28:0/920292156 wait complete. 2024-09-19T09:34:37.014 INFO:tasks.workunit.client.0.smithi028.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-19T09:34:37.014 INFO:tasks.workunit.client.0.smithi028.stdout:checking if more special flags are required for pthreads... no 2024-09-19T09:34:37.014 INFO:tasks.workunit.client.0.smithi028.stdout:checking for cc_r... gcc 2024-09-19T09:34:37.081 INFO:tasks.workunit.client.0.smithi028.stdout:checking for setrlimit... yes 2024-09-19T09:34:37.143 INFO:tasks.workunit.client.0.smithi028.stdout:checking for setproctitle... no 2024-09-19T09:34:37.207 INFO:tasks.workunit.client.0.smithi028.stdout:checking for getopt_long... yes 2024-09-19T09:34:37.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:37 smithi028 ceph-mon[30972]: pgmap v396: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-09-19T09:34:37.281 INFO:tasks.workunit.client.0.smithi028.stdout:checking for strtoull... yes 2024-09-19T09:34:37.346 INFO:tasks.workunit.client.0.smithi028.stdout:checking for strtoq... yes 2024-09-19T09:34:37.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:37 smithi078 ceph-mon[35619]: pgmap v396: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-09-19T09:34:37.412 INFO:tasks.workunit.client.0.smithi028.stdout:checking for snprintf... yes 2024-09-19T09:34:37.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:37 smithi038 ceph-mon[35271]: pgmap v396: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-09-19T09:34:37.476 INFO:tasks.workunit.client.0.smithi028.stdout:checking for vsnprintf... yes 2024-09-19T09:34:37.543 INFO:tasks.workunit.client.0.smithi028.stdout:checking for vfprintf... yes 2024-09-19T09:34:37.605 INFO:tasks.workunit.client.0.smithi028.stdout:checking for gethostname... yes 2024-09-19T09:34:37.669 INFO:tasks.workunit.client.0.smithi028.stdout:checking for setlocale... yes 2024-09-19T09:34:37.734 INFO:tasks.workunit.client.0.smithi028.stdout:checking for sync... yes 2024-09-19T09:34:37.798 INFO:tasks.workunit.client.0.smithi028.stdout:checking whether snprintf is C99 conformant... done 2024-09-19T09:34:37.809 INFO:tasks.workunit.client.0.smithi028.stdout:configure: creating ./config.status 2024-09-19T09:34:37.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee32d0c4', total seconds elapsed: 23.6 2024-09-19T09:34:37.962 INFO:tasks.workunit.client.0.smithi028.stdout:config.status: creating Makefile 2024-09-19T09:34:37.980 INFO:tasks.workunit.client.0.smithi028.stdout:config.status: creating src/Makefile 2024-09-19T09:34:38.001 INFO:tasks.workunit.client.0.smithi028.stdout:config.status: creating man/Makefile 2024-09-19T09:34:38.015 INFO:tasks.workunit.client.0.smithi028.stdout:config.status: creating config.h 2024-09-19T09:34:38.074 INFO:tasks.workunit.client.0.smithi028.stdout:config.status: executing depfiles commands 2024-09-19T09:34:38.207 INFO:tasks.workunit.client.0.smithi028.stderr:+ make 2024-09-19T09:34:38.217 INFO:tasks.workunit.client.0.smithi028.stdout:make all-recursive 2024-09-19T09:34:38.220 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T09:34:38.222 INFO:tasks.workunit.client.0.smithi028.stdout:Making all in src 2024-09-19T09:34:38.226 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-19T09:34:38.226 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-19T09:34:38.226 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-19T09:34:38.336 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-19T09:34:38.337 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-19T09:34:38.404 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-19T09:34:38.404 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-19T09:34:38.458 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-19T09:34:38.458 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-19T09:34:38.509 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-19T09:34:38.509 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-19T09:34:38.561 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-19T09:34:38.561 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-19T09:34:38.611 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-19T09:34:38.612 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-19T09:34:38.687 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-19T09:34:38.687 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-19T09:34:38.723 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-19T09:34:38.723 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-19T09:34:38.757 INFO:tasks.workunit.client.0.smithi028.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-19T09:34:38.757 INFO:tasks.workunit.client.0.smithi028.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-19T09:34:38.795 INFO:tasks.workunit.client.0.smithi028.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-19T09:34:38.814 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-19T09:34:38.814 INFO:tasks.workunit.client.0.smithi028.stdout:Making all in man 2024-09-19T09:34:38.817 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-19T09:34:38.817 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Nothing to be done for 'all'. 2024-09-19T09:34:38.817 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-19T09:34:38.818 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T09:34:38.819 INFO:tasks.workunit.client.0.smithi028.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T09:34:38.819 INFO:tasks.workunit.client.0.smithi028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T09:34:38.820 INFO:tasks.workunit.client.0.smithi028.stderr:+ cd src 2024-09-19T09:34:38.820 INFO:tasks.workunit.client.0.smithi028.stderr:+ mkdir blogtest_in 2024-09-19T09:34:38.822 INFO:tasks.workunit.client.0.smithi028.stdout:running blogbench 2024-09-19T09:34:38.822 INFO:tasks.workunit.client.0.smithi028.stderr:+ echo 'running blogbench' 2024-09-19T09:34:38.822 INFO:tasks.workunit.client.0.smithi028.stderr:+ ./blogbench -d blogtest_in 2024-09-19T09:34:38.823 INFO:tasks.workunit.client.0.smithi028.stdout: 2024-09-19T09:34:38.823 INFO:tasks.workunit.client.0.smithi028.stdout:Frequency = 10 secs 2024-09-19T09:34:38.823 INFO:tasks.workunit.client.0.smithi028.stdout:Scratch dir = [blogtest_in] 2024-09-19T09:34:39.036 INFO:tasks.workunit.client.0.smithi028.stdout:Spawning 3 writers... 2024-09-19T09:34:39.036 INFO:tasks.workunit.client.0.smithi028.stdout:Spawning 1 rewriters... 2024-09-19T09:34:39.036 INFO:tasks.workunit.client.0.smithi028.stdout:Spawning 5 commenters... 2024-09-19T09:34:39.036 INFO:tasks.workunit.client.0.smithi028.stdout:Spawning 100 readers... 2024-09-19T09:34:39.038 INFO:tasks.workunit.client.0.smithi028.stdout:Benchmarking for 30 iterations. 2024-09-19T09:34:39.038 INFO:tasks.workunit.client.0.smithi028.stdout:The test will run during 5 minutes. 2024-09-19T09:34:39.038 INFO:tasks.workunit.client.0.smithi028.stdout: 2024-09-19T09:34:39.038 INFO:tasks.workunit.client.0.smithi028.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-19T09:34:39.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:39 smithi078 ceph-mon[35619]: pgmap v397: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-19T09:34:39.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:39 smithi038 ceph-mon[35271]: pgmap v397: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-19T09:34:39.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:39 smithi028 ceph-mon[30972]: pgmap v397: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-19T09:34:41.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:41 smithi078 ceph-mon[35619]: pgmap v398: 97 pgs: 97 active+clean; 4.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-19T09:34:41.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:41 smithi038 ceph-mon[35271]: pgmap v398: 97 pgs: 97 active+clean; 4.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-19T09:34:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:41 smithi028 ceph-mon[30972]: pgmap v398: 97 pgs: 97 active+clean; 4.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-19T09:34:43.356 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:43 smithi078 ceph-mon[35619]: pgmap v399: 97 pgs: 97 active+clean; 7.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 316 KiB/s wr, 22 op/s 2024-09-19T09:34:43.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:43 smithi038 ceph-mon[35271]: pgmap v399: 97 pgs: 97 active+clean; 7.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 316 KiB/s wr, 22 op/s 2024-09-19T09:34:43.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:43 smithi028 ceph-mon[30972]: pgmap v399: 97 pgs: 97 active+clean; 7.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 316 KiB/s wr, 22 op/s 2024-09-19T09:34:45.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:45 smithi038 ceph-mon[35271]: pgmap v400: 97 pgs: 97 active+clean; 8.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 505 KiB/s wr, 33 op/s 2024-09-19T09:34:45.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:45 smithi028 ceph-mon[30972]: pgmap v400: 97 pgs: 97 active+clean; 8.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 505 KiB/s wr, 33 op/s 2024-09-19T09:34:45.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:45 smithi078 ceph-mon[35619]: pgmap v400: 97 pgs: 97 active+clean; 8.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 505 KiB/s wr, 33 op/s 2024-09-19T09:34:46.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:46 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:46.513 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:46 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:46.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:46 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:47.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:47 smithi038 ceph-mon[35271]: pgmap v401: 97 pgs: 97 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 791 KiB/s wr, 49 op/s 2024-09-19T09:34:47.425 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:47 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:34:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:47 smithi028 ceph-mon[30972]: pgmap v401: 97 pgs: 97 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 791 KiB/s wr, 49 op/s 2024-09-19T09:34:47.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:47 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.532+0000 7f236e93b640 1 -- 172.21.15.28:0/3040844056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23681025f0 msgr2=0x7f2368108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.532+0000 7f236e93b640 1 --2- 172.21.15.28:0/3040844056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23681025f0 0x7f2368108e80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f236002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 -- 172.21.15.28:0/3040844056 shutdown_connections 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 --2- 172.21.15.28:0/3040844056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23681025f0 0x7f2368108e80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 --2- 172.21.15.28:0/3040844056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2368101c30 0x7f23681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 --2- 172.21.15.28:0/3040844056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368100a40 0x7f2368100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 -- 172.21.15.28:0/3040844056 >> 172.21.15.28:0/3040844056 conn(0x7f23680fc1c0 msgr2=0x7f23680fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 -- 172.21.15.28:0/3040844056 shutdown_connections 2024-09-19T09:34:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 -- 172.21.15.28:0/3040844056 wait complete. 2024-09-19T09:34:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 Processor -- start 2024-09-19T09:34:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.533+0000 7f236e93b640 1 -- start start 2024-09-19T09:34:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 0x7f236806a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23681025f0 0x7f236819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f236810c6c0 con 0x7f2368100a40 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f236810c540 con 0x7f2368101c30 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236e93b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f236810c840 con 0x7f23681025f0 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 0x7f236806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f236ced8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23681025f0 0x7f236819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:60620/0 (socket says 172.21.15.28:60620) 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 -- 172.21.15.28:0/311599309 learned_addr learned my addr 172.21.15.28:0/311599309 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 -- 172.21.15.28:0/311599309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23681025f0 msgr2=0x7f236819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23681025f0 0x7f236819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 -- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 msgr2=0x7f236806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 0x7f236806a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f23677fe640 1 -- 172.21.15.28:0/311599309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236819faa0 con 0x7f2368101c30 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.534+0000 7f2367fff640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 0x7f236806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f23677fe640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e9f0 tx=0x7f235c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f23657fa640 1 -- 172.21.15.28:0/311599309 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c00cd80 con 0x7f2368101c30 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236819fd30 con 0x7f2368101c30 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f23657fa640 1 -- 172.21.15.28:0/311599309 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c005500 con 0x7f2368101c30 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f23657fa640 1 -- 172.21.15.28:0/311599309 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c010690 con 0x7f2368101c30 2024-09-19T09:34:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.535+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a0220 con 0x7f2368101c30 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.537+0000 7f23716d5740 1 Processor -- start 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.537+0000 7f23716d5740 1 -- start start 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.537+0000 7f23716d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23716d5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23716d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb4a370 0x557e0bb4c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23716d5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557e0baa82f0 con 0x557e0bb452e0 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23716d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e0ba50270 con 0x557e0bb4a370 2024-09-19T09:34:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23716d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e0baa42b0 con 0x557e0bb449a0 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35988/0 (socket says 172.21.15.28:35988) 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 -- 172.21.15.28:0/3323937674 learned_addr learned my addr 172.21.15.28:0/3323937674 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f23657fa640 1 -- 172.21.15.28:0/311599309 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f235c0046c0 con 0x7f2368101c30 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 -- 172.21.15.28:0/3323937674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb4a370 msgr2=0x557e0bb4c830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb4a370 0x557e0bb4c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 -- 172.21.15.28:0/3323937674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 msgr2=0x557e0bb49da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb49da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 -- 172.21.15.28:0/3323937674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0ba4f340 con 0x557e0bb449a0 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f2367fff640 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.538+0000 7f236ced8640 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f2360004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3a4102b61c6a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f234e7fc640 1 -- 172.21.15.28:0/3323937674 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600098c0 con 0x557e0bb449a0 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f234e7fc640 1 -- 172.21.15.28:0/3323937674 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2360036040 con 0x557e0bb449a0 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f234e7fc640 1 -- 172.21.15.28:0/3323937674 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360004290 con 0x557e0bb449a0 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23657fa640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 0x7f23500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f2367fff640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 0x7f23500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23657fa640 1 -- 172.21.15.28:0/311599309 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f235c0cfcf0 con 0x7f2368101c30 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 -- 172.21.15.28:0/3323937674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 msgr2=0x557e0bb44da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f2360004a40 comp rx=0 tx=0).stop 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 -- 172.21.15.28:0/3323937674 shutdown_connections 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb4a370 0x557e0bb4c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 --2- 172.21.15.28:0/3323937674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb449a0 0x557e0bb44da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 -- 172.21.15.28:0/3323937674 >> 172.21.15.28:0/3323937674 conn(0x557e0bb40010 msgr2=0x557e0bb40410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f23716d5740 1 -- 172.21.15.28:0/3323937674 shutdown_connections 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.539+0000 7f2367fff640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 0x7f23500a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f2358005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 -- 172.21.15.28:0/3323937674 wait complete. 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 -- start start 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 Processor -- start 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb449a0 0x557e0bb4ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb55f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557e0bb4d100 con 0x557e0bb452e0 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e0ba31290 con 0x557e0bb449a0 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.540+0000 7f23716d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e0bb4d280 con 0x557e0bb4a370 2024-09-19T09:34:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23716d5740 10 client.0 _ll_register_callbacks cb 0x557e0ba62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50570/0 (socket says 172.21.15.28:50570) 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 -- 172.21.15.28:0/3977490520 learned_addr learned my addr 172.21.15.28:0/3977490520 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f2367fff640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb449a0 0x557e0bb4ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f236ced8640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb55f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 -- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 msgr2=0x557e0bb55f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb55f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 -- 172.21.15.28:0/3977490520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb449a0 msgr2=0x557e0bb4ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb449a0 0x557e0bb4ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 -- 172.21.15.28:0/3977490520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0b7bb210 con 0x557e0bb452e0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23677fe640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f235c098740 tx=0x7f235c09bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caab7b85d5caa71d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23477fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0a4cc0 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f235c0a0170 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2360036040 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f235c0a4f50 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23716d5740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.541+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e0b887620 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2360036040 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f235c09b830 con 0x557e0bb452e0 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 20 client.16071 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bb5eb30 0x557e0bb60f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 4 client.16071 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e0bab5a20 con 0x557e0bb5eb30 2024-09-19T09:34:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f2367fff640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bb5eb30 0x557e0bb60f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f2367fff640 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bb5eb30 0x557e0bb60f20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b8eda47512942e server_cookie=4838e7b5ff066d4a in_seq=0 out_seq=0 2024-09-19T09:34:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.542+0000 7f23477fe640 10 client.16071 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23477fe640 1 -- 172.21.15.28:0/3977490520 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e0bab5a20 con 0x557e0bb5eb30 2024-09-19T09:34:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23477fe640 10 client.16071 handle_command_reply: tid=0 2024-09-19T09:34:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 client.16071 shutdown 2024-09-19T09:34:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 msgr2=0x557e0bb4efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f235c098740 tx=0x7f235c09bd00 comp rx=0 tx=0).stop 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 shutdown_connections 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bb5eb30 0x557e0bb60f20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb55f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23477fe640 0 client.16071 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb452e0 0x557e0bb4efb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 --2- 172.21.15.28:0/3977490520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb449a0 0x557e0bb4ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 >> 172.21.15.28:0/3977490520 conn(0x557e0bb40010 msgr2=0x557e0bb41820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 shutdown_connections 2024-09-19T09:34:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 1 -- 172.21.15.28:0/3977490520 wait complete. 2024-09-19T09:34:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.546+0000 7f23716d5740 20 client.16071 trim_cache size 0 max 16384 2024-09-19T09:34:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 Processor -- start 2024-09-19T09:34:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 -- start start 2024-09-19T09:34:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb55f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a6c60 con 0x557e0bb449a0 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a87d0 con 0x557e0bb452e0 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23716d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a7330 con 0x557e0bb4a370 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50580/0 (socket says 172.21.15.28:50580) 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f2367fff640 1 -- 172.21.15.28:0/1819880028 learned_addr learned my addr 172.21.15.28:0/1819880028 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb55f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f236ced8640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 -- 172.21.15.28:0/1819880028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 msgr2=0x557e0bb41400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb41400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 -- 172.21.15.28:0/1819880028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 msgr2=0x557e0bb40ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f23677fe640 1 -- 172.21.15.28:0/1819880028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0bb61290 con 0x557e0bb452e0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.562+0000 7f2367fff640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23677fe640 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb55f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f235c098c60 tx=0x7f235c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fd74304fb8cd10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f234dffb640 1 -- 172.21.15.28:0/1819880028 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c098240 con 0x557e0bb452e0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f234dffb640 1 -- 172.21.15.28:0/1819880028 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c096e50 con 0x557e0bb452e0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f234dffb640 1 -- 172.21.15.28:0/1819880028 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c09b260 con 0x557e0bb452e0 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 -- 172.21.15.28:0/1819880028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 msgr2=0x557e0bb55f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb55f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f235c098c60 tx=0x7f235c00cf90 comp rx=0 tx=0).stop 2024-09-19T09:34:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 -- 172.21.15.28:0/1819880028 shutdown_connections 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb55f40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb41400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 --2- 172.21.15.28:0/1819880028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb40ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 -- 172.21.15.28:0/1819880028 >> 172.21.15.28:0/1819880028 conn(0x557e0bb40010 msgr2=0x557e0bb46560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 -- 172.21.15.28:0/1819880028 shutdown_connections 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.563+0000 7f23716d5740 1 -- 172.21.15.28:0/1819880028 wait complete. 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 -- start start 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 Processor -- start 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb53bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb54c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a80d0 con 0x557e0bb449a0 2024-09-19T09:34:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a7a00 con 0x557e0bb452e0 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23681a4c40 con 0x557e0bb4a370 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23716d5740 10 client.0 _ll_register_callbacks cb 0x7f235c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f23677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb53bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f2367fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50594/0 (socket says 172.21.15.28:50594) 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f236ced8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.564+0000 7f2367fff640 1 -- 172.21.15.28:0/3602943054 learned_addr learned my addr 172.21.15.28:0/3602943054 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 -- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 msgr2=0x557e0bb54c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb54c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 -- 172.21.15.28:0/3602943054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 msgr2=0x557e0bb536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 -- 172.21.15.28:0/3602943054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0bb55210 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23677fe640 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb53bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f235c098c60 tx=0x7f235c09a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3d161f2bff5f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0a4d40 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f235c096e50 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f235c0a0170 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f235c0a0370 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 10 client.25697 fetch_fsmap learned FSMap version 28 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e0bb55210 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c096e50 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f235c004f40 con 0x557e0bb452e0 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 10 client.25697 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 20 client.25697 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bbc9e30 0x557e0bbcc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 4 client.25697 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:34:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.565+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e0baafd70 con 0x557e0bbc9e30 2024-09-19T09:34:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.566+0000 7f2367fff640 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bbc9e30 0x557e0bbcc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.566+0000 7f23477fe640 10 client.25697 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:47.566+0000 7f2367fff640 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bbc9e30 0x557e0bbcc250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e30efcae4dbeaf4 server_cookie=8f065ba018bbba80 in_seq=0 out_seq=0 2024-09-19T09:34:47.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:47 smithi078 ceph-mon[35619]: pgmap v401: 97 pgs: 97 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 791 KiB/s wr, 49 op/s 2024-09-19T09:34:47.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:47 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:34:48.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23477fe640 1 -- 172.21.15.28:0/3602943054 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e0baafd70 con 0x557e0bbc9e30 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23477fe640 10 client.25697 handle_command_reply: tid=0 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 client.25697 shutdown 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 msgr2=0x557e0bb53bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb53bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f235c098c60 tx=0x7f235c09a7f0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 shutdown_connections 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e0bb452e0 0x557e0bb53bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557e0bbc9e30 0x557e0bbcc250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e0bb4a370 0x557e0bb54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 --2- 172.21.15.28:0/3602943054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e0bb449a0 0x557e0bb536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23477fe640 0 client.25697 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 >> 172.21.15.28:0/3602943054 conn(0x557e0bb40010 msgr2=0x557e0bb3e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 shutdown_connections 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 1 -- 172.21.15.28:0/3602943054 wait complete. 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.656+0000 7f23716d5740 20 client.25697 trim_cache size 0 max 16384 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:34:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:34:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 msgr2=0x7f23500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 0x7f23500a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f2358005f90 comp rx=0 tx=0).stop 2024-09-19T09:34:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 msgr2=0x7f236819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e9f0 tx=0x7f235c00edf0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 shutdown_connections 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f23500a3240 0x7f23500a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368101c30 0x7f236819cb70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23681025f0 0x7f236819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 --2- 172.21.15.28:0/311599309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2368100a40 0x7f236806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 >> 172.21.15.28:0/311599309 conn(0x7f23680fc1c0 msgr2=0x7f2368105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 shutdown_connections 2024-09-19T09:34:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:34:48.657+0000 7f236e93b640 1 -- 172.21.15.28:0/311599309 wait complete. 2024-09-19T09:34:48.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.9 MiB/s wr, 106 op/s 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:34:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.676 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.9 MiB/s wr, 106 op/s 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:34:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:34:48.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.9 MiB/s wr, 106 op/s 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:34:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:34:49.038 INFO:tasks.workunit.client.0.smithi028.stdout: 15 165681 929 91576 728 35764 1178 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 -- 172.21.15.28:0/2549745560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c8102d50 msgr2=0x7f71c8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 --2- 172.21.15.28:0/2549745560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c8102d50 0x7f71c8103130 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009900 tx=0x7f71b802a480 comp rx=0 tx=0).stop 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 -- 172.21.15.28:0/2549745560 shutdown_connections 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 --2- 172.21.15.28:0/2549745560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8103670 0x7f71c810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 --2- 172.21.15.28:0/2549745560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c810c180 0x7f71c810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 --2- 172.21.15.28:0/2549745560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c8102d50 0x7f71c8103130 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 -- 172.21.15.28:0/2549745560 >> 172.21.15.28:0/2549745560 conn(0x7f71c80fc7d0 msgr2=0x7f71c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 -- 172.21.15.28:0/2549745560 shutdown_connections 2024-09-19T09:34:49.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.134+0000 7f71cd5db640 1 -- 172.21.15.28:0/2549745560 wait complete. 2024-09-19T09:34:49.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 Processor -- start 2024-09-19T09:34:49.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 -- start start 2024-09-19T09:34:49.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 0x7f71c8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 0x7f71c819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8111f80 con 0x7f71c810c180 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8111e00 con 0x7f71c8102d50 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71cd5db640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8112100 con 0x7f71c8103670 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:60662/0 (socket says 172.21.15.28:60662) 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 0x7f71c819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71c6ffd640 1 -- 172.21.15.28:0/1863897230 learned_addr learned my addr 172.21.15.28:0/1863897230 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.135+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 0x7f71c8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 -- 172.21.15.28:0/1863897230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 msgr2=0x7f71c8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 0x7f71c8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 -- 172.21.15.28:0/1863897230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 msgr2=0x7f71c8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 -- 172.21.15.28:0/1863897230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c819d3f0 con 0x7f71c810c180 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:49.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 0x7f71c8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 0x7f71c819cd10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009fd0 tx=0x7f71c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71affff640 1 -- 172.21.15.28:0/1863897230 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c000ce20 con 0x7f71c810c180 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c819d6e0 con 0x7f71c810c180 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71affff640 1 -- 172.21.15.28:0/1863897230 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c0005020 con 0x7f71c810c180 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.136+0000 7f71affff640 1 -- 172.21.15.28:0/1863897230 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0010720 con 0x7f71c810c180 2024-09-19T09:34:49.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.137+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a4fc0 con 0x7f71c810c180 2024-09-19T09:34:49.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.139+0000 7f71affff640 1 -- 172.21.15.28:0/1863897230 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f71c0005540 con 0x7f71c810c180 2024-09-19T09:34:49.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.139+0000 7f71cf9b4740 1 Processor -- start 2024-09-19T09:34:49.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.139+0000 7f71cf9b4740 1 -- start start 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71affff640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 0x7f71b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e931140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e931680 0x55b58e933a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71affff640 1 -- 172.21.15.28:0/1863897230 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f71c00cfe70 con 0x7f71c810c180 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b58e88c2e0 con 0x55b58e9264a0 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b58e833ce0 con 0x55b58e931680 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b58e8882a0 con 0x55b58e926dc0 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 0x7f71b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50626/0 (socket says 172.21.15.28:50626) 2024-09-19T09:34:49.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 -- 172.21.15.28:0/1207984808 learned_addr learned my addr 172.21.15.28:0/1207984808 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e931680 0x55b58e933a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e931140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 -- 172.21.15.28:0/1207984808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 msgr2=0x55b58e931140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e931140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 -- 172.21.15.28:0/1207984808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e931680 msgr2=0x55b58e933a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e931680 0x55b58e933a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.140+0000 7f71c6ffd640 1 -- 172.21.15.28:0/1207984808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58e818440 con 0x55b58e9264a0 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.141+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 0x7f71b00a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71c810d550 tx=0x7f71bc013c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.141+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009a20 tx=0x7f71b8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74d5a23d17fe7be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.141+0000 7f71ad7fa640 1 -- 172.21.15.28:0/1207984808 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8035040 con 0x55b58e9264a0 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.141+0000 7f71ad7fa640 1 -- 172.21.15.28:0/1207984808 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b80318f0 con 0x55b58e9264a0 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.141+0000 7f71ad7fa640 1 -- 172.21.15.28:0/1207984808 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b803fa40 con 0x55b58e9264a0 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1207984808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 msgr2=0x55b58e926880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009a20 tx=0x7f71b8005dd0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1207984808 shutdown_connections 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e931680 0x55b58e933a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e931140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1207984808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e9264a0 0x55b58e926880 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1207984808 >> 172.21.15.28:0/1207984808 conn(0x55b58e922a30 msgr2=0x55b58e922e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1207984808 shutdown_connections 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1207984808 wait complete. 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 -- start start 2024-09-19T09:34:49.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.142+0000 7f71cf9b4740 1 Processor -- start 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 0x55b58e92e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b58e934250 con 0x55b58e926dc0 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b58e847050 con 0x55b58e9264a0 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b58e9343d0 con 0x55b58e931680 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71cf9b4740 10 client.0 _ll_register_callbacks cb 0x55b58e887160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 0x55b58e92e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50636/0 (socket says 172.21.15.28:50636) 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 -- 172.21.15.28:0/499668269 learned_addr learned my addr 172.21.15.28:0/499668269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:60680/0 (socket says 172.21.15.28:60680) 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 -- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 msgr2=0x55b58e92e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 0x55b58e92e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 -- 172.21.15.28:0/499668269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 msgr2=0x55b58e930fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 -- 172.21.15.28:0/499668269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58e5fd3c0 con 0x55b58e926dc0 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c77fe640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 0x55b58e92e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71c67fc640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00fd20 tx=0x7f71bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15137edf918f6260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.143+0000 7f71aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc024030 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f71bc003af0 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b80318f0 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71bc003cf0 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 10 client.16089 fetch_fsmap learned FSMap version 28 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b58e93fc00 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b80318f0 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f71bc01f070 con 0x55b58e926dc0 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 10 client.16089 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 20 client.16089 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e941110 0x55b58e943590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 4 client.16089 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:49.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.144+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b58e899850 con 0x55b58e941110 2024-09-19T09:34:49.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.145+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e941110 0x55b58e943590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.145+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e941110 0x55b58e943590 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c11f721cebcb83 server_cookie=b13ad7ac57626059 in_seq=0 out_seq=0 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.145+0000 7f71aaff5640 10 client.16089 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71aaff5640 1 -- 172.21.15.28:0/499668269 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b58e899850 con 0x55b58e941110 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71aaff5640 10 client.16089 handle_command_reply: tid=0 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 client.16089 shutdown 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 msgr2=0x55b58e92e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00fd20 tx=0x7f71bc004730 comp rx=0 tx=0).stop 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 shutdown_connections 2024-09-19T09:34:49.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e941110 0x55b58e943590 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e931680 0x55b58e92e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e926dc0 0x55b58e92e260 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/499668269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e930fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 >> 172.21.15.28:0/499668269 conn(0x55b58e922a30 msgr2=0x55b58e928710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71aaff5640 0 client.16089 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 shutdown_connections 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 1 -- 172.21.15.28:0/499668269 wait complete. 2024-09-19T09:34:49.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.149+0000 7f71cf9b4740 20 client.16089 trim_cache size 0 max 16384 2024-09-19T09:34:49.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 Processor -- start 2024-09-19T09:34:49.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 -- start start 2024-09-19T09:34:49.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 0x55b58e9301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e926dc0 0x55b58e948e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ab2e0 con 0x55b58e931680 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ace50 con 0x55b58e926dc0 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.168+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ab9b0 con 0x55b58e9264a0 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 0x55b58e9301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 0x55b58e9301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36060/0 (socket says 172.21.15.28:36060) 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c6ffd640 1 -- 172.21.15.28:0/3886741453 learned_addr learned my addr 172.21.15.28:0/3886741453 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50648/0 (socket says 172.21.15.28:50648) 2024-09-19T09:34:49.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c67fc640 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e926dc0 0x55b58e948e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 -- 172.21.15.28:0/3886741453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 msgr2=0x55b58e9301d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 0x55b58e9301d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 -- 172.21.15.28:0/3886741453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e926dc0 msgr2=0x55b58e948e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e926dc0 0x55b58e948e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 -- 172.21.15.28:0/3886741453 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58e92c7f0 con 0x55b58e931680 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71c77fe640 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f71c001f000 tx=0x7f71c00cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7076df658d049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.169+0000 7f71acff9640 1 -- 172.21.15.28:0/3886741453 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b9070 con 0x55b58e931680 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71acff9640 1 -- 172.21.15.28:0/3886741453 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00a0870 con 0x55b58e931680 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71acff9640 1 -- 172.21.15.28:0/3886741453 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00cde60 con 0x55b58e931680 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 -- 172.21.15.28:0/3886741453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 msgr2=0x55b58e9493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f71c001f000 tx=0x7f71c00cf040 comp rx=0 tx=0).stop 2024-09-19T09:34:49.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 -- 172.21.15.28:0/3886741453 shutdown_connections 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e926dc0 0x55b58e948e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9493a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/3886741453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e9264a0 0x55b58e9301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 -- 172.21.15.28:0/3886741453 >> 172.21.15.28:0/3886741453 conn(0x55b58e922a30 msgr2=0x55b58e92fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 -- 172.21.15.28:0/3886741453 shutdown_connections 2024-09-19T09:34:49.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.170+0000 7f71cf9b4740 1 -- 172.21.15.28:0/3886741453 wait complete. 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 -- start start 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 Processor -- start 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e936d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ac750 con 0x55b58e931680 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81ac080 con 0x55b58e9264a0 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81a92c0 con 0x55b58e926dc0 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71cf9b4740 10 client.0 _ll_register_callbacks cb 0x7f71bc013550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:49.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71c67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71c77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71c67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36074/0 (socket says 172.21.15.28:36074) 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71c67fc640 1 -- 172.21.15.28:0/1178489128 learned_addr learned my addr 172.21.15.28:0/1178489128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.171+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e936d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 -- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 msgr2=0x55b58e9372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 -- 172.21.15.28:0/1178489128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 msgr2=0x55b58e936d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e936d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 -- 172.21.15.28:0/1178489128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58e9a44e0 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c67fc640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71c77fe640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9a3e70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f71c00a52f0 tx=0x7f71c0005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ee2560e9f9d559 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b9070 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f71c00051d0 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00a0870 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71c00cde60 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71cf9b4740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.172+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b58e9ab850 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00a0870 con 0x55b58e931680 2024-09-19T09:34:49.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f71c009b380 con 0x55b58e931680 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 20 client.16095 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e9abf00 0x55b58e9ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 4 client.16095 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b58e893d60 con 0x55b58e9abf00 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.173+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e9abf00 0x55b58e9ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.174+0000 7f71c6ffd640 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e9abf00 0x55b58e9ae320 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de329e5021a8692e server_cookie=a8e15dd0a4298a30 in_seq=0 out_seq=0 2024-09-19T09:34:49.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.174+0000 7f71aaff5640 10 client.16095 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:49.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71aaff5640 1 -- 172.21.15.28:0/1178489128 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b58e893d60 con 0x55b58e9abf00 2024-09-19T09:34:49.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71aaff5640 10 client.16095 handle_command_reply: tid=0 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 client.16095 shutdown 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 msgr2=0x55b58e9a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9a3e70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f71c00a52f0 tx=0x7f71c0005940 comp rx=0 tx=0).stop 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 shutdown_connections 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b58e9abf00 0x55b58e9ae320 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71aaff5640 0 client.16095 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b58e931680 0x55b58e9a3e70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b58e926dc0 0x55b58e9372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 --2- 172.21.15.28:0/1178489128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b58e9264a0 0x55b58e936d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 >> 172.21.15.28:0/1178489128 conn(0x55b58e922a30 msgr2=0x55b58e9302e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 shutdown_connections 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 1 -- 172.21.15.28:0/1178489128 wait complete. 2024-09-19T09:34:49.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.178+0000 7f71cf9b4740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T09:34:49.184 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:34:49.184 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2024-09-19T09:34:49.184 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2024-09-19T09:34:49.184 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 msgr2=0x7f71b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 0x7f71b00a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71c810d550 tx=0x7f71bc013c10 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 msgr2=0x7f71c819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 0x7f71c819cd10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009fd0 tx=0x7f71c000ef10 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 shutdown_connections 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f71b00a3240 0x7f71b00a5700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71c810c180 0x7f71c819cd10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71c8103670 0x7f71c8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 --2- 172.21.15.28:0/1863897230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71c8102d50 0x7f71c8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:49.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 >> 172.21.15.28:0/1863897230 conn(0x7f71c80fc7d0 msgr2=0x7f71c80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:49.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 shutdown_connections 2024-09-19T09:34:49.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:49.179+0000 7f71cd5db640 1 -- 172.21.15.28:0/1863897230 wait complete. 2024-09-19T09:34:50.435 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:34:50.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:34:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:50.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:50 smithi028 ceph-mon[30972]: pgmap v403: 97 pgs: 97 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-09-19T09:34:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:50 smithi078 ceph-mon[35619]: pgmap v403: 97 pgs: 97 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-09-19T09:34:50.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[65760]: *** Caught signal (Segmentation fault) ** 2024-09-19T09:34:50.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[65760]: in thread 7fedffb56640 thread_name:MR_Finisher 2024-09-19T09:34:50.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:50 smithi078 podman[68960]: 2024-09-19 09:34:50.622578122 +0000 UTC m=+0.021553493 container died 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-19T09:34:50.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:50 smithi038 ceph-mon[35271]: pgmap v403: 97 pgs: 97 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-09-19T09:34:51.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.137+0000 7f41fc747640 1 -- 172.21.15.28:0/1139493388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4100e20 msgr2=0x7f41f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:51.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.137+0000 7f41fc747640 1 --2- 172.21.15.28:0/1139493388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4100e20 0x7f41f4101200 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009900 tx=0x7f41f002a480 comp rx=0 tx=0).stop 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 -- 172.21.15.28:0/1139493388 shutdown_connections 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 --2- 172.21.15.28:0/1139493388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4102ef0 0x7f41f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 --2- 172.21.15.28:0/1139493388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102570 0x7f41f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 --2- 172.21.15.28:0/1139493388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4100e20 0x7f41f4101200 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 -- 172.21.15.28:0/1139493388 >> 172.21.15.28:0/1139493388 conn(0x7f41f40fc7d0 msgr2=0x7f41f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 -- 172.21.15.28:0/1139493388 shutdown_connections 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 -- 172.21.15.28:0/1139493388 wait complete. 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 Processor -- start 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.138+0000 7f41fc747640 1 -- start start 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:51.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102ef0 0x7f41f419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4110fd0 con 0x7f41f4102570 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4110e50 con 0x7f41f4100e20 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fc747640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4111150 con 0x7f41f4102ef0 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41f9ce2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41face4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102ef0 0x7f41f419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:60700/0 (socket says 172.21.15.28:60700) 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41f9ce2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50682/0 (socket says 172.21.15.28:50682) 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 -- 172.21.15.28:0/662839272 learned_addr learned my addr 172.21.15.28:0/662839272 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 -- 172.21.15.28:0/662839272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102ef0 msgr2=0x7f41f419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102ef0 0x7f41f419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 -- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 msgr2=0x7f41f4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:51.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 -- 172.21.15.28:0/662839272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f419d510 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41f9ce2640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.139+0000 7f41fa4e3640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f002aa70 tx=0x7f41f0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.140+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f003f070 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.140+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f419d7a0 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.140+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0031c30 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.140+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0035030 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.140+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41a50e0 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.141+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f4101200 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.142+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f41f0042070 con 0x7f41f4100e20 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.143+0000 7f41eb7fe640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.143+0000 7f41f9ce2640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:51.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.143+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f41f00f0600 con 0x7f41f4100e20 2024-09-19T09:34:51.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.143+0000 7f41f9ce2640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 0x7f41e00a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41f40ffd20 tx=0x7f41e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:51.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.146+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41f0031de0 con 0x7f41f4100e20 2024-09-19T09:34:51.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.373+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41f41a52b0 con 0x7f41f4100e20 2024-09-19T09:34:51.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.373+0000 7f41eb7fe640 1 -- 172.21.15.28:0/662839272 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f41f00bcd00 con 0x7f41f4100e20 2024-09-19T09:34:51.376 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:34:51.376 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":29,"btime":"2024-09-19T09:34:50:509282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:34:49.725446+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2992347158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2992347158},{"type":"v1","addr":"172.21.15.78:6833","nonce":2992347158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:34:51.376 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 29 2024-09-19T09:34:51.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 msgr2=0x7f41e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:51.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 0x7f41e00a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41f40ffd20 tx=0x7f41e4005f90 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 msgr2=0x7f41f4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f002aa70 tx=0x7f41f0003c10 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 shutdown_connections 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41f4102ef0 0x7f41f419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f4102570 0x7f41f4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 --2- 172.21.15.28:0/662839272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41f4100e20 0x7f41f4198560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 >> 172.21.15.28:0/662839272 conn(0x7f41f40fc7d0 msgr2=0x7f41f406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 shutdown_connections 2024-09-19T09:34:51.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:51.376+0000 7f41fc747640 1 -- 172.21.15.28:0/662839272 wait complete. 2024-09-19T09:34:51.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:51 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:51.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:51 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/662839272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:51.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:51 smithi078 podman[68960]: 2024-09-19 09:34:51.345727309 +0000 UTC m=+0.744702662 container remove 0c2a7c0bcdd6778a7bba81e9af837a85175525a03a3847ec274f5d886bda19f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T09:34:51.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:51 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-09-19T09:34:51.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:51 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:51.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:51 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/662839272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:51.958 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:51 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:34:51.958 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:51 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/662839272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:34:51.986 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:34:52.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:51 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-09-19T09:34:52.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:34:51 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Consumed 2.749s CPU time. 2024-09-19T09:34:52.250 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:52.682 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:52 smithi028 ceph-mon[30972]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-19T09:34:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- 172.21.15.28:0/102703323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 msgr2=0x7f8f9010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 --2- 172.21.15.28:0/102703323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f9010d950 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009cd0 tx=0x7f8f8802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- 172.21.15.28:0/102703323 shutdown_connections 2024-09-19T09:34:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 --2- 172.21.15.28:0/102703323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f9010d950 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 --2- 172.21.15.28:0/102703323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 0x7f8f90100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 --2- 172.21.15.28:0/102703323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 0x7f8f90106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- 172.21.15.28:0/102703323 >> 172.21.15.28:0/102703323 conn(0x7f8f900fc7a0 msgr2=0x7f8f900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- 172.21.15.28:0/102703323 shutdown_connections 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- 172.21.15.28:0/102703323 wait complete. 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 Processor -- start 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.680+0000 7f8f97c58640 1 -- start start 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 0x7f8f900fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 0x7f8f900fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90110e80 con 0x7f8f90101420 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90110d00 con 0x7f8f90106a80 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f97c58640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90111000 con 0x7f8f90100a80 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50706/0 (socket says 172.21.15.28:50706) 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 -- 172.21.15.28:0/1905765165 learned_addr learned my addr 172.21.15.28:0/1905765165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f959f4640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 0x7f8f900fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 0x7f8f900fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 -- 172.21.15.28:0/1905765165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 msgr2=0x7f8f900fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 0x7f8f900fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 -- 172.21.15.28:0/1905765165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 msgr2=0x7f8f900fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 0x7f8f900fd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 -- 172.21.15.28:0/1905765165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f901a68e0 con 0x7f8f90101420 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f951f3640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e9f0 tx=0x7f8f8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.681+0000 7f8f86ffd640 1 -- 172.21.15.28:0/1905765165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000ccc0 con 0x7f8f90101420 2024-09-19T09:34:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.682+0000 7f8f86ffd640 1 -- 172.21.15.28:0/1905765165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8000ce60 con 0x7f8f90101420 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.682+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f901a6bd0 con 0x7f8f90101420 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.682+0000 7f8f86ffd640 1 -- 172.21.15.28:0/1905765165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80010690 con 0x7f8f90101420 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.682+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f901a7110 con 0x7f8f90101420 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f86ffd640 1 -- 172.21.15.28:0/1905765165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8f800046c0 con 0x7f8f90101420 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 Processor -- start 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 -- start start 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d101465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101470370 0x55d101472760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d1013cb2f0 con 0x55d101466b40 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d101372cf0 con 0x55d101470370 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.684+0000 7f8f9a031740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1013c72b0 con 0x55d1014655d0 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f961f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d101465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36106/0 (socket says 172.21.15.28:36106) 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 -- 172.21.15.28:0/3623732168 learned_addr learned my addr 172.21.15.28:0/3623732168 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f951f3640 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101470370 0x55d101472760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 -- 172.21.15.28:0/3623732168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101470370 msgr2=0x55d101472760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101470370 0x55d101472760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f86ffd640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 0x7f8f7c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 -- 172.21.15.28:0/3623732168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 msgr2=0x55d101465090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d101465090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 -- 172.21.15.28:0/3623732168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1013573a0 con 0x55d1014655d0 2024-09-19T09:34:52.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 0x7f8f7c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f86ffd640 1 -- 172.21.15.28:0/1905765165 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f800cfd10 con 0x7f8f90101420 2024-09-19T09:34:52.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f959f4640 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00ac90 tx=0x7f8f8c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1c263fc7b9a33d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.685+0000 7f8f7bfff640 1 -- 172.21.15.28:0/3623732168 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01d070 con 0x55d1014655d0 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f7bfff640 1 -- 172.21.15.28:0/3623732168 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8c018b20 con 0x55d1014655d0 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f7bfff640 1 -- 172.21.15.28:0/3623732168 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c010b40 con 0x55d1014655d0 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 -- 172.21.15.28:0/3623732168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 msgr2=0x55d101465a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00ac90 tx=0x7f8f8c00f790 comp rx=0 tx=0).stop 2024-09-19T09:34:52.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 0x7f8f7c0a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009430 tx=0x7f8f8803f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 -- 172.21.15.28:0/3623732168 shutdown_connections 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101470370 0x55d101472760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d101465a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 --2- 172.21.15.28:0/3623732168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d101465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 -- 172.21.15.28:0/3623732168 >> 172.21.15.28:0/3623732168 conn(0x55d10142e310 msgr2=0x55d101461c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 -- 172.21.15.28:0/3623732168 shutdown_connections 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.686+0000 7f8f9a031740 1 -- 172.21.15.28:0/3623732168 wait complete. 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 -- start start 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 Processor -- start 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d10146fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d10146d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d101473070 con 0x55d101466b40 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d101386060 con 0x55d1014655d0 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1014731f0 con 0x55d101470370 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f9a031740 10 client.0 _ll_register_callbacks cb 0x55d1010d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50730/0 (socket says 172.21.15.28:50730) 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f951f3640 1 -- 172.21.15.28:0/2278284786 learned_addr learned my addr 172.21.15.28:0/2278284786 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.687+0000 7f8f961f5640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d10146d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f959f4640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d10146fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 -- 172.21.15.28:0/2278284786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 msgr2=0x55d10146d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d10146d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 -- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 msgr2=0x55d10146fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d10146fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 -- 172.21.15.28:0/2278284786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10113c3c0 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f951f3640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e9c0 tx=0x7f8f8009bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789f9ceb8b71a28c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f797fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800a4d40 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f8009b7b0 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c018b20 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f800a0710 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f9a031740 10 client.16113 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d10147eb40 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.688+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8c018b20 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f8f800a0170 con 0x55d101466b40 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:52.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 10 client.16113 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 20 client.16113 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d10147ff50 0x55d101482380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 4 client.16113 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1013d8860 con 0x55d10147ff50 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.689+0000 7f8f959f4640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d10147ff50 0x55d101482380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f797fa640 10 client.16113 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f959f4640 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d10147ff50 0x55d101482380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2197f236f6de8d02 server_cookie=adbc5b0b9b0bf339 in_seq=0 out_seq=0 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f797fa640 1 -- 172.21.15.28:0/2278284786 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1013d8860 con 0x55d10147ff50 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f797fa640 10 client.16113 handle_command_reply: tid=0 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f9a031740 1 client.16113 shutdown 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 msgr2=0x55d10146d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.690+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e9c0 tx=0x7f8f8009bf60 comp rx=0 tx=0).stop 2024-09-19T09:34:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 shutdown_connections 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d10147ff50 0x55d101482380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d10146d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d10146d020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 --2- 172.21.15.28:0/2278284786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d10146fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f797fa640 0 client.16113 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 >> 172.21.15.28:0/2278284786 conn(0x55d10142e310 msgr2=0x55d101467930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 shutdown_connections 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 1 -- 172.21.15.28:0/2278284786 wait complete. 2024-09-19T09:34:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.691+0000 7f8f9a031740 20 client.16113 trim_cache size 0 max 16384 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 Processor -- start 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 -- start start 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101466b40 0x55d101468120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 0x55d101468660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901af600 con 0x55d101470370 2024-09-19T09:34:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901b1200 con 0x55d101466b40 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f9a031740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901afcd0 con 0x55d1014655d0 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f961f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 0x55d101468660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36118/0 (socket says 172.21.15.28:36118) 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f961f5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 0x55d101468660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50736/0 (socket says 172.21.15.28:50736) 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 -- 172.21.15.28:0/952392879 learned_addr learned my addr 172.21.15.28:0/952392879 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f951f3640 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101466b40 0x55d101468120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 -- 172.21.15.28:0/952392879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101466b40 msgr2=0x55d101468120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101466b40 0x55d101468120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 -- 172.21.15.28:0/952392879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 msgr2=0x55d101468660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 0x55d101468660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.705+0000 7f8f959f4640 1 -- 172.21.15.28:0/952392879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d101462dd0 con 0x55d1014655d0 2024-09-19T09:34:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f959f4640 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00abb0 tx=0x7f8f8c0106c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a087ddadf059b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f7b7fe640 1 -- 172.21.15.28:0/952392879 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c032070 con 0x55d1014655d0 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f7b7fe640 1 -- 172.21.15.28:0/952392879 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8c005520 con 0x55d1014655d0 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f7b7fe640 1 -- 172.21.15.28:0/952392879 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01d040 con 0x55d1014655d0 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 -- 172.21.15.28:0/952392879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 msgr2=0x55d10146ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00abb0 tx=0x7f8f8c0106c0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 -- 172.21.15.28:0/952392879 shutdown_connections 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d101466b40 0x55d101468120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101470370 0x55d101468660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 --2- 172.21.15.28:0/952392879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1014655d0 0x55d10146ef90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.706+0000 7f8f9a031740 1 -- 172.21.15.28:0/952392879 >> 172.21.15.28:0/952392879 conn(0x55d10142e310 msgr2=0x55d10146e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- 172.21.15.28:0/952392879 shutdown_connections 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- 172.21.15.28:0/952392879 wait complete. 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- start start 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 Processor -- start 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d1014784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d1014789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d101475480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901b0a70 con 0x55d101466b40 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901b03a0 con 0x55d1014655d0 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901ad5e0 con 0x55d101470370 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.707+0000 7f8f9a031740 10 client.0 _ll_register_callbacks cb 0x7f8f80009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d1014789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d101475480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f951f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d1014789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50744/0 (socket says 172.21.15.28:50744) 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f951f3640 1 -- 172.21.15.28:0/1464239509 learned_addr learned my addr 172.21.15.28:0/1464239509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f959f4640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d1014784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 -- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 msgr2=0x55d1014784b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d1014784b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 -- 172.21.15.28:0/1464239509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 msgr2=0x55d1014789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d1014789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 -- 172.21.15.28:0/1464239509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1014759c0 con 0x55d101470370 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f959f4640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d1014784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f961f5640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d101475480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006010 tx=0x7f8f88005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3bec49773c2160a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f797fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88057060 con 0x55d101470370 2024-09-19T09:34:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c005520 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.708+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8f88004610 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f88004810 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 10 client.25675 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1014eb2b0 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8c005520 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f8f88051070 con 0x55d101470370 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 10 client.25675 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 10 client.25675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 20 client.25675 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d1014eb9b0 0x55d1014eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 4 client.25675 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1013d2d70 con 0x55d1014eb9b0 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.709+0000 7f8f959f4640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d1014eb9b0 0x55d1014eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f797fa640 10 client.25675 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f959f4640 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d1014eb9b0 0x55d1014eddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56bcfe9e48ee14e3 server_cookie=fd32ac4e1ed5d00c in_seq=0 out_seq=0 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f797fa640 1 -- 172.21.15.28:0/1464239509 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1013d2d70 con 0x55d1014eb9b0 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f797fa640 10 client.25675 handle_command_reply: tid=0 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 client.25675 shutdown 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 msgr2=0x55d101475480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d101475480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006010 tx=0x7f8f88005c50 comp rx=0 tx=0).stop 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 shutdown_connections 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x55d1014eb9b0 0x55d1014eddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d101470370 0x55d101475480 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d101466b40 0x55d1014789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f797fa640 0 client.25675 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 --2- 172.21.15.28:0/1464239509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d1014655d0 0x55d1014784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 >> 172.21.15.28:0/1464239509 conn(0x55d10142e310 msgr2=0x55d10146f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 shutdown_connections 2024-09-19T09:34:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 1 -- 172.21.15.28:0/1464239509 wait complete. 2024-09-19T09:34:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.710+0000 7f8f9a031740 20 client.25675 trim_cache size 0 max 16384 2024-09-19T09:34:52.729 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 msgr2=0x7f8f7c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 0x7f8f7c0a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009430 tx=0x7f8f8803f040 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 msgr2=0x7f8f900fd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e9f0 tx=0x7f8f8000edf0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 shutdown_connections 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8f7c0a32c0 0x7f8f7c0a5780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f90106a80 0x7f8f900fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f90101420 0x7f8f900fd200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 --2- 172.21.15.28:0/1905765165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f90100a80 0x7f8f900fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 >> 172.21.15.28:0/1905765165 conn(0x7f8f900fc7a0 msgr2=0x7f8f9006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 shutdown_connections 2024-09-19T09:34:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:52.711+0000 7f8f97c58640 1 -- 172.21.15.28:0/1905765165 wait complete. 2024-09-19T09:34:52.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:52 smithi078 ceph-mon[35619]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-19T09:34:52.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:52 smithi038 ceph-mon[35271]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 440 op/s 2024-09-19T09:34:53.495 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:34:53.758 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:54.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 -- 172.21.15.28:0/2608924978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c102d40 msgr2=0x7f229c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 --2- 172.21.15.28:0/2608924978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c102d40 0x7f229c103120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802a480 comp rx=0 tx=0).stop 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 -- 172.21.15.28:0/2608924978 shutdown_connections 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 --2- 172.21.15.28:0/2608924978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c103660 0x7f229c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 --2- 172.21.15.28:0/2608924978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c10c170 0x7f229c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 --2- 172.21.15.28:0/2608924978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c102d40 0x7f229c103120 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 -- 172.21.15.28:0/2608924978 >> 172.21.15.28:0/2608924978 conn(0x7f229c0fc7a0 msgr2=0x7f229c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 -- 172.21.15.28:0/2608924978 shutdown_connections 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.198+0000 7f22a3288640 1 -- 172.21.15.28:0/2608924978 wait complete. 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 Processor -- start 2024-09-19T09:34:54.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 -- start start 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c102d40 0x7f229c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 0x7f229c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f229c111fd0 con 0x7f229c10c170 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f229c111e50 con 0x7f229c102d40 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a3288640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f229c112150 con 0x7f229c103660 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42994/0 (socket says 172.21.15.28:42994) 2024-09-19T09:34:54.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 -- 172.21.15.28:0/537472512 learned_addr learned my addr 172.21.15.28:0/537472512 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1024640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c102d40 0x7f229c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a0823640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 0x7f229c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 -- 172.21.15.28:0/537472512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 msgr2=0x7f229c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 0x7f229c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 -- 172.21.15.28:0/537472512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c102d40 msgr2=0x7f229c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c102d40 0x7f229c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.199+0000 7f22a1825640 1 -- 172.21.15.28:0/537472512 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c1a6990 con 0x7f229c10c170 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22a0823640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 0x7f229c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22a1825640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f229400eb10 tx=0x7f229400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22927fc640 1 -- 172.21.15.28:0/537472512 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294004b20 con 0x7f229c10c170 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22927fc640 1 -- 172.21.15.28:0/537472512 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2294004cc0 con 0x7f229c10c170 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c1a6c80 con 0x7f229c10c170 2024-09-19T09:34:54.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22927fc640 1 -- 172.21.15.28:0/537472512 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294010760 con 0x7f229c10c170 2024-09-19T09:34:54.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.200+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a71c0 con 0x7f229c10c170 2024-09-19T09:34:54.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22927fc640 1 -- 172.21.15.28:0/537472512 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2294005020 con 0x7f229c10c170 2024-09-19T09:34:54.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 Processor -- start 2024-09-19T09:34:54.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 -- start start 2024-09-19T09:34:54.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cda01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f9cd95a2f0 con 0x55f9cd9ff370 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f9cd901cf0 con 0x55f9cd9f5b40 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a5661740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9cd9562b0 con 0x55f9cd9f45d0 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a1024640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a0823640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cda01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a1024640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45326/0 (socket says 172.21.15.28:45326) 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a1024640 1 -- 172.21.15.28:0/3791937917 learned_addr learned my addr 172.21.15.28:0/3791937917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22a0823640 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.203+0000 7f22927fc640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 0x7f22880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 -- 172.21.15.28:0/3791937917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 msgr2=0x55f9cd9f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 -- 172.21.15.28:0/3791937917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 msgr2=0x55f9cda01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cda01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 -- 172.21.15.28:0/3791937917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9cd8e63a0 con 0x55f9cd9f45d0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1825640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 0x7f22880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22927fc640 1 -- 172.21.15.28:0/537472512 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f22940cff90 con 0x7f229c10c170 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1024640 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d1f7e4948d58b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22877fe640 1 -- 172.21.15.28:0/3791937917 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298040070 con 0x55f9cd9f45d0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22a1825640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 0x7f22880a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f229406f4d0 tx=0x7f22940c7450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22877fe640 1 -- 172.21.15.28:0/3791937917 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2298031970 con 0x55f9cd9f45d0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.204+0000 7f22877fe640 1 -- 172.21.15.28:0/3791937917 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298035040 con 0x55f9cd9f45d0 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- 172.21.15.28:0/3791937917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 msgr2=0x55f9cd9f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298005e80 comp rx=0 tx=0).stop 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- 172.21.15.28:0/3791937917 shutdown_connections 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cda01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f4a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 --2- 172.21.15.28:0/3791937917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- 172.21.15.28:0/3791937917 >> 172.21.15.28:0/3791937917 conn(0x55f9cd9bd310 msgr2=0x55f9cd9f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- 172.21.15.28:0/3791937917 shutdown_connections 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- 172.21.15.28:0/3791937917 wait complete. 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.205+0000 7f22a5661740 1 -- start start 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 Processor -- start 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f9cda02010 con 0x55f9cd9f5b40 2024-09-19T09:34:54.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f9cd915060 con 0x55f9cd9f45d0 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9cda02190 con 0x55f9cd9ff370 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a5661740 10 client.0 _ll_register_callbacks cb 0x55f9cd6688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a1024640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a1024640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43022/0 (socket says 172.21.15.28:43022) 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a1024640 1 -- 172.21.15.28:0/3212478507 learned_addr learned my addr 172.21.15.28:0/3212478507 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a0823640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.206+0000 7f22a0823640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 -- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 msgr2=0x55f9cd9fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 -- 172.21.15.28:0/3212478507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 msgr2=0x55f9cd9fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9fecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 -- 172.21.15.28:0/3212478507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9cd6cb3c0 con 0x55f9cd9f5b40 2024-09-19T09:34:54.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a1024640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2298031520 tx=0x7f22980052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71058fbe842bc4be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f2284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298040030 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2298004260 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2298031970 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2298035040 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a5661740 10 client.16131 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9cda0da50 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.207+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2298031970 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f22980036f0 con 0x55f9cd9f5b40 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 10 client.16131 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 10 client.16131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 20 client.16131 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda0eda0 0x55f9cda11190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 4 client.16131 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9cd967860 con 0x55f9cda0eda0 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a0823640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda0eda0 0x55f9cda11190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f2284ff9640 10 client.16131 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.208+0000 7f22a0823640 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda0eda0 0x55f9cda11190 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4279aaa179b1d3f server_cookie=9e7968a1efa592ed in_seq=0 out_seq=0 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f2284ff9640 1 -- 172.21.15.28:0/3212478507 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9cd967860 con 0x55f9cda0eda0 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f2284ff9640 10 client.16131 handle_command_reply: tid=0 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 client.16131 shutdown 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 msgr2=0x55f9cd9fbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2298031520 tx=0x7f22980052d0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 shutdown_connections 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda0eda0 0x55f9cda11190 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cd9fbf70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 --2- 172.21.15.28:0/3212478507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 >> 172.21.15.28:0/3212478507 conn(0x55f9cd9bd310 msgr2=0x55f9cd9f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.209+0000 7f2284ff9640 0 client.16131 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.210+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 shutdown_connections 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.210+0000 7f22a5661740 1 -- 172.21.15.28:0/3212478507 wait complete. 2024-09-19T09:34:54.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.210+0000 7f22a5661740 20 client.16131 trim_cache size 0 max 16384 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 Processor -- start 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 -- start start 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cda0b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1af740 con 0x55f9cd9ff370 2024-09-19T09:34:54.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1b1340 con 0x55f9cd9f5b40 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a5661740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1afe10 con 0x55f9cd9f45d0 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.223+0000 7f22a0823640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a0823640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a0823640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45352/0 (socket says 172.21.15.28:45352) 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a0823640 1 -- 172.21.15.28:0/1483888138 learned_addr learned my addr 172.21.15.28:0/1483888138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cda0b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 -- 172.21.15.28:0/1483888138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 msgr2=0x55f9cd9f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 -- 172.21.15.28:0/1483888138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 msgr2=0x55f9cd9f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 -- 172.21.15.28:0/1483888138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9cd9fc840 con 0x55f9cd9f5b40 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a0823640 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f22a1024640 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cda0b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2298005560 tx=0x7f2298005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c07eda628e95139 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f2286ffd640 1 -- 172.21.15.28:0/1483888138 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298040070 con 0x55f9cd9f5b40 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f2286ffd640 1 -- 172.21.15.28:0/1483888138 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229804d070 con 0x55f9cd9f5b40 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.224+0000 7f2286ffd640 1 -- 172.21.15.28:0/1483888138 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298003850 con 0x55f9cd9f5b40 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- 172.21.15.28:0/1483888138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 msgr2=0x55f9cda0b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cda0b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2298005560 tx=0x7f2298005e80 comp rx=0 tx=0).stop 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- 172.21.15.28:0/1483888138 shutdown_connections 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f5b40 0x55f9cda0b2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 --2- 172.21.15.28:0/1483888138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9f45d0 0x55f9cd9f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- 172.21.15.28:0/1483888138 >> 172.21.15.28:0/1483888138 conn(0x55f9cd9bd310 msgr2=0x55f9cd9f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- 172.21.15.28:0/1483888138 shutdown_connections 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- 172.21.15.28:0/1483888138 wait complete. 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.225+0000 7f22a5661740 1 -- start start 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 Processor -- start 2024-09-19T09:34:54.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1b0bb0 con 0x55f9cd9f5b40 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1b04e0 con 0x55f9cd9f45d0 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1ad740 con 0x55f9cd9ff370 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a5661740 10 client.0 _ll_register_callbacks cb 0x7f229c06b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a1024640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a0823640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a0823640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:53414/0 (socket says 172.21.15.28:53414) 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a1024640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43046/0 (socket says 172.21.15.28:43046) 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a0823640 1 -- 172.21.15.28:0/1753000746 learned_addr learned my addr 172.21.15.28:0/1753000746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.226+0000 7f22a0823640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 -- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 msgr2=0x55f9cd9f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 -- 172.21.15.28:0/1753000746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 msgr2=0x55f9cda07960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 -- 172.21.15.28:0/1753000746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9cd9f8a60 con 0x55f9cd9f45d0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a1024640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a0823640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f229c102430 tx=0x7f228c0047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21aaaa6210c1075 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f2284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c00f040 con 0x55f9cd9f45d0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229804d070 con 0x55f9cd9f45d0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f228c017d90 con 0x55f9cd9f45d0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f228c036020 con 0x55f9cd9f45d0 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a5661740 10 client.25751 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9cda78f80 con 0x55f9cd9f45d0 2024-09-19T09:34:54.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.227+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229804d070 con 0x55f9cd9f45d0 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f228c015040 con 0x55f9cd9f45d0 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 10 client.25751 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 10 client.25751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 20 client.25751 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda79660 0x55f9cda7ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 4 client.25751 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9cd961d70 con 0x55f9cda79660 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a1024640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda79660 0x55f9cda7ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f22a1024640 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda79660 0x55f9cda7ba80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2ba87d89d71d29a server_cookie=7d45dc5675f3bd3f in_seq=0 out_seq=0 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.228+0000 7f2284ff9640 10 client.25751 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f2284ff9640 1 -- 172.21.15.28:0/1753000746 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9cd961d70 con 0x55f9cda79660 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f2284ff9640 10 client.25751 handle_command_reply: tid=0 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 client.25751 shutdown 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 msgr2=0x55f9cda07420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f229c102430 tx=0x7f228c0047d0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 shutdown_connections 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f9cda79660 0x55f9cda7ba80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9cd9ff370 0x55f9cd9f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f9cd9f5b40 0x55f9cda07960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f2284ff9640 0 client.25751 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 --2- 172.21.15.28:0/1753000746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f9cd9f45d0 0x55f9cda07420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 >> 172.21.15.28:0/1753000746 conn(0x55f9cd9bd310 msgr2=0x55f9cd9fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 shutdown_connections 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 1 -- 172.21.15.28:0/1753000746 wait complete. 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.229+0000 7f22a5661740 20 client.25751 trim_cache size 0 max 16384 2024-09-19T09:34:54.236 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 msgr2=0x7f22880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 0x7f22880a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f229406f4d0 tx=0x7f22940c7450 comp rx=0 tx=0).stop 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 msgr2=0x7f229c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f229400eb10 tx=0x7f229400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 shutdown_connections 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f22880a3240 0x7f22880a5700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f229c10c170 0x7f229c101420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f229c103660 0x7f229c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 --2- 172.21.15.28:0/537472512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229c102d40 0x7f229c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:54.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 >> 172.21.15.28:0/537472512 conn(0x7f229c0fc7a0 msgr2=0x7f229c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:54.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 shutdown_connections 2024-09-19T09:34:54.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:54.230+0000 7f22a3288640 1 -- 172.21.15.28:0/537472512 wait complete. 2024-09-19T09:34:54.856 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:54 smithi078 ceph-mon[35619]: pgmap v405: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 629 op/s 2024-09-19T09:34:54.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:54 smithi038 ceph-mon[35271]: pgmap v405: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 629 op/s 2024-09-19T09:34:55.013 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:54 smithi028 ceph-mon[30972]: pgmap v405: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 629 op/s 2024-09-19T09:34:55.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:34:55.375 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1223261652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0101de0 msgr2=0x7fbbb01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1223261652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0101de0 0x7fbbb01021c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009900 tx=0x7fbbac02a410 comp rx=0 tx=0).stop 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1223261652 shutdown_connections 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1223261652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0102790 0x7fbbb01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1223261652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0106f10 0x7fbbb010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1223261652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0101de0 0x7fbbb01021c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.829+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1223261652 >> 172.21.15.28:0/1223261652 conn(0x7fbbb00fd8c0 msgr2=0x7fbbb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1223261652 shutdown_connections 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1223261652 wait complete. 2024-09-19T09:34:55.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 Processor -- start 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- start start 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0101de0 0x7fbbb00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0102790 0x7fbbb0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0111f70 con 0x7fbbb0106f10 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0111df0 con 0x7fbbb0101de0 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.830+0000 7fbbb7e27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01120f0 con 0x7fbbb0102790 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43064/0 (socket says 172.21.15.28:43064) 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 -- 172.21.15.28:0/1657176421 learned_addr learned my addr 172.21.15.28:0/1657176421 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0102790 0x7fbbb0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0101de0 0x7fbbb00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 -- 172.21.15.28:0/1657176421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0102790 msgr2=0x7fbbb0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0102790 0x7fbbb0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 -- 172.21.15.28:0/1657176421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0101de0 msgr2=0x7fbbb00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0101de0 0x7fbbb00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 -- 172.21.15.28:0/1657176421 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01a6990 con 0x7fbbb0106f10 2024-09-19T09:34:55.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb63c4640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009fd0 tx=0x7fbba800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbba6ffd640 1 -- 172.21.15.28:0/1657176421 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8017070 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbba6ffd640 1 -- 172.21.15.28:0/1657176421 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba8003e50 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01a6c80 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbba6ffd640 1 -- 172.21.15.28:0/1657176421 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba80106c0 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.831+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a71c0 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.834+0000 7fbba6ffd640 1 -- 172.21.15.28:0/1657176421 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbba8003970 con 0x7fbbb0106f10 2024-09-19T09:34:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.834+0000 7fbbba200740 1 Processor -- start 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.834+0000 7fbbba200740 1 -- start start 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55fbca2262f0 con 0x55fbca2c05d0 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbca1cdcf0 con 0x55fbca2c1b40 2024-09-19T09:34:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbba200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fbca2222b0 con 0x55fbca2cb370 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb53c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbba6ffd640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 0x7fbb9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43068/0 (socket says 172.21.15.28:43068) 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/156194434 learned_addr learned my addr 172.21.15.28:0/156194434 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb63c4640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 0x7fbb9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbba6ffd640 1 -- 172.21.15.28:0/1657176421 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbba80cf530 con 0x7fbbb0106f10 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/156194434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 msgr2=0x55fbca2cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/156194434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 msgr2=0x55fbca2c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.835+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/156194434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbca1b23a0 con 0x55fbca2c05d0 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.836+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009900 tx=0x7fbbac031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8355f2fef26d433 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.836+0000 7fbbb63c4640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 0x7fbb9c0a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbba809bb60 tx=0x7fbba8066610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.836+0000 7fbb9bfff640 1 -- 172.21.15.28:0/156194434 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac040070 con 0x55fbca2c05d0 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.836+0000 7fbb9bfff640 1 -- 172.21.15.28:0/156194434 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbac005020 con 0x55fbca2c05d0 2024-09-19T09:34:55.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.836+0000 7fbb9bfff640 1 -- 172.21.15.28:0/156194434 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac035040 con 0x55fbca2c05d0 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- 172.21.15.28:0/156194434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 msgr2=0x55fbca2c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009900 tx=0x7fbbac031630 comp rx=0 tx=0).stop 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- 172.21.15.28:0/156194434 shutdown_connections 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c05d0 0x55fbca2c0a10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 --2- 172.21.15.28:0/156194434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- 172.21.15.28:0/156194434 >> 172.21.15.28:0/156194434 conn(0x55fbca289310 msgr2=0x55fbca2bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- 172.21.15.28:0/156194434 shutdown_connections 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- 172.21.15.28:0/156194434 wait complete. 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 -- start start 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.837+0000 7fbbba200740 1 Processor -- start 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2cb370 0x55fbca2c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55fbca2ce010 con 0x55fbca2cb370 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fbca1e1060 con 0x55fbca2c1b40 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fbca2ce190 con 0x55fbca2c05d0 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbba200740 10 client.0 _ll_register_callbacks cb 0x55fbc9f348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45392/0 (socket says 172.21.15.28:45392) 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 -- 172.21.15.28:0/2349366270 learned_addr learned my addr 172.21.15.28:0/2349366270 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2cb370 0x55fbca2c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 -- 172.21.15.28:0/2349366270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 msgr2=0x55fbca2c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 -- 172.21.15.28:0/2349366270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2cb370 msgr2=0x55fbca2c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2cb370 0x55fbca2c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb53c2640 1 -- 172.21.15.28:0/2349366270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc9f973c0 con 0x55fbca2c05d0 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.838+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010ea0 tx=0x7fbba00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a344736f38f7015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0005020 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbba00051c0 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac005020 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbba00053c0 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fbca2d9a80 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbac005020 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fbba00224f0 con 0x55fbca2c05d0 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 10 client.25705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 20 client.25705 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca2dae90 0x55fbca2dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 4 client.25705 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.839+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbca233860 con 0x55fbca2dae90 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.840+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca2dae90 0x55fbca2dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.840+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca2dae90 0x55fbca2dd2a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afd017cdce580d14 server_cookie=c9aff66357f540fc in_seq=0 out_seq=0 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.840+0000 7fbb997fa640 10 client.25705 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbb997fa640 1 -- 172.21.15.28:0/2349366270 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbca233860 con 0x55fbca2dae90 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbb997fa640 10 client.25705 handle_command_reply: tid=0 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 client.25705 shutdown 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 msgr2=0x55fbca2cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010ea0 tx=0x7fbba00046c0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 shutdown_connections 2024-09-19T09:34:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca2dae90 0x55fbca2dd2a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c1b40 0x55fbca2c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2cb370 0x55fbca2c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 --2- 172.21.15.28:0/2349366270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2c05d0 0x55fbca2cac90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbb997fa640 0 client.25705 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 >> 172.21.15.28:0/2349366270 conn(0x55fbca289310 msgr2=0x55fbca2c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 shutdown_connections 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 1 -- 172.21.15.28:0/2349366270 wait complete. 2024-09-19T09:34:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.841+0000 7fbbba200740 20 client.25705 trim_cache size 0 max 16384 2024-09-19T09:34:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 Processor -- start 2024-09-19T09:34:55.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 -- start start 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01af640 con 0x55fbca2c1b40 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b1240 con 0x55fbca2c05d0 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbba200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01afd10 con 0x55fbca2cb370 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43082/0 (socket says 172.21.15.28:43082) 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/4002612090 learned_addr learned my addr 172.21.15.28:0/4002612090 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/4002612090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 msgr2=0x55fbca2c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/4002612090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 msgr2=0x55fbca2bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/4002612090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbca2c6ef0 con 0x55fbca2c1b40 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.855+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0096c0 tx=0x7fbbac005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3826c5b61298e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbb9b7fe640 1 -- 172.21.15.28:0/4002612090 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac053070 con 0x55fbca2c1b40 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbb9b7fe640 1 -- 172.21.15.28:0/4002612090 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbac004240 con 0x55fbca2c1b40 2024-09-19T09:34:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbb9b7fe640 1 -- 172.21.15.28:0/4002612090 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac035040 con 0x55fbca2c1b40 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 -- 172.21.15.28:0/4002612090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 msgr2=0x55fbca2c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0096c0 tx=0x7fbbac005600 comp rx=0 tx=0).stop 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 -- 172.21.15.28:0/4002612090 shutdown_connections 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c3120 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 --2- 172.21.15.28:0/4002612090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.856+0000 7fbbba200740 1 -- 172.21.15.28:0/4002612090 >> 172.21.15.28:0/4002612090 conn(0x55fbca289310 msgr2=0x55fbca2bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- 172.21.15.28:0/4002612090 shutdown_connections 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- 172.21.15.28:0/4002612090 wait complete. 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- start start 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 Processor -- start 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b0ab0 con 0x55fbca2c1b40 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b03e0 con 0x55fbca2c05d0 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01ad640 con 0x55fbca2cb370 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbba200740 10 client.0 _ll_register_callbacks cb 0x7fbba000ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbb5bc3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43090/0 (socket says 172.21.15.28:43090) 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.857+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/3333121195 learned_addr learned my addr 172.21.15.28:0/3333121195 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 msgr2=0x55fbca2c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/3333121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 msgr2=0x55fbca2d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2d3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 -- 172.21.15.28:0/3333121195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbca344b40 con 0x55fbca2c1b40 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbb5bc3640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02a930 tx=0x7fbbac02ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f061f863a24ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbb997fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac035070 con 0x55fbca2c1b40 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fbbac003c50 con 0x55fbca2c1b40 2024-09-19T09:34:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac004240 con 0x55fbca2c1b40 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbac05f070 con 0x55fbca2c1b40 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbba200740 10 client.16155 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.858+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fbca345c90 con 0x55fbca2c1b40 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbac004240 con 0x55fbca2c1b40 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fbbac058790 con 0x55fbca2c1b40 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 10 client.16155 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 10 client.16155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 20 client.16155 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca345ec0 0x55fbca3482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 4 client.16155 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbca22dd70 con 0x55fbca345ec0 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca345ec0 0x55fbca3482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:55.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbbb53c2640 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca345ec0 0x55fbca3482b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=197b98dc9cf3a2ad server_cookie=6c7320e0c31c19e1 in_seq=0 out_seq=0 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.859+0000 7fbb997fa640 10 client.16155 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbb997fa640 1 -- 172.21.15.28:0/3333121195 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbca22dd70 con 0x55fbca345ec0 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbb997fa640 10 client.16155 handle_command_reply: tid=0 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 client.16155 shutdown 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 msgr2=0x55fbca2c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02a930 tx=0x7fbbac02ab50 comp rx=0 tx=0).stop 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 shutdown_connections 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55fbca345ec0 0x55fbca3482b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fbca2cb370 0x55fbca2c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fbca2c1b40 0x55fbca2c4de0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 --2- 172.21.15.28:0/3333121195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fbca2c05d0 0x55fbca2d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 >> 172.21.15.28:0/3333121195 conn(0x55fbca289310 msgr2=0x55fbca2bbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbb997fa640 0 client.16155 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:34:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 shutdown_connections 2024-09-19T09:34:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 1 -- 172.21.15.28:0/3333121195 wait complete. 2024-09-19T09:34:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.860+0000 7fbbba200740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T09:34:55.868 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 msgr2=0x7fbb9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 0x7fbb9c0a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbba809bb60 tx=0x7fbba8066610 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 msgr2=0x7fbbb0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009fd0 tx=0x7fbba800b260 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 shutdown_connections 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fbb9c0a32c0 0x7fbb9c0a5780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbb0106f10 0x7fbbb0074f70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbb0102790 0x7fbbb0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 --2- 172.21.15.28:0/1657176421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb0101de0 0x7fbbb00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 >> 172.21.15.28:0/1657176421 conn(0x7fbbb00fd8c0 msgr2=0x7fbbb00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 shutdown_connections 2024-09-19T09:34:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:55.861+0000 7fbbb7e27640 1 -- 172.21.15.28:0/1657176421 wait complete. 2024-09-19T09:34:56.587 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:34:56.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:56 smithi028 ceph-mon[30972]: pgmap v406: 97 pgs: 97 active+clean; 45 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 621 op/s 2024-09-19T09:34:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:56.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:56 smithi078 ceph-mon[35619]: pgmap v406: 97 pgs: 97 active+clean; 45 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 621 op/s 2024-09-19T09:34:56.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:56 smithi038 ceph-mon[35271]: pgmap v406: 97 pgs: 97 active+clean; 45 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 621 op/s 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 -- 172.21.15.28:0/1457190881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 msgr2=0x7f9eb41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1457190881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb41067e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea002a410 comp rx=0 tx=0).stop 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 -- 172.21.15.28:0/1457190881 shutdown_connections 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1457190881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1457190881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb41067e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1457190881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 0x7f9eb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 -- 172.21.15.28:0/1457190881 >> 172.21.15.28:0/1457190881 conn(0x7f9eb40fd8c0 msgr2=0x7f9eb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 -- 172.21.15.28:0/1457190881 shutdown_connections 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.339+0000 7f9ebafac640 1 -- 172.21.15.28:0/1457190881 wait complete. 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.340+0000 7f9ebafac640 1 Processor -- start 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.340+0000 7f9ebafac640 1 -- start start 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.340+0000 7f9ebafac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4111f30 con 0x7f9eb4102790 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4111db0 con 0x7f9eb4106f10 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41120b0 con 0x7f9eb4101de0 2024-09-19T09:34:57.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43110/0 (socket says 172.21.15.28:43110) 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1821994004 learned_addr learned my addr 172.21.15.28:0/1821994004 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb9549640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1821994004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 msgr2=0x7f9eb4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1821994004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 msgr2=0x7f9eb419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1821994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb419d4c0 con 0x7f9eb4102790 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb9549640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006010 tx=0x7f9ea00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb1ffb640 1 -- 172.21.15.28:0/1821994004 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003f030 con 0x7f9eb4102790 2024-09-19T09:34:57.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9eb1ffb640 1 -- 172.21.15.28:0/1821994004 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea0003ce0 con 0x7f9eb4102790 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.341+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb419d750 con 0x7f9eb4102790 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.342+0000 7f9eb1ffb640 1 -- 172.21.15.28:0/1821994004 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0035070 con 0x7f9eb4102790 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.342+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb41a5090 con 0x7f9eb4102790 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9eb1ffb640 1 -- 172.21.15.28:0/1821994004 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9ea0042030 con 0x7f9eb4102790 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 Processor -- start 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 -- start start 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282c6780 0x5588282c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55882821d2f0 con 0x5588282b75d0 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588281c4cf0 con 0x5588282c6780 2024-09-19T09:34:57.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.345+0000 7f9ebd385740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588282192b0 con 0x5588282b8b40 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb1ffb640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 0x7f9ea40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43116/0 (socket says 172.21.15.28:43116) 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb9549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 -- 172.21.15.28:0/3791591639 learned_addr learned my addr 172.21.15.28:0/3791591639 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282c6780 0x5588282c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb9549640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 0x7f9ea40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb1ffb640 1 -- 172.21.15.28:0/1821994004 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9ea00effa0 con 0x7f9eb4102790 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 -- 172.21.15.28:0/3791591639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 msgr2=0x5588282b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 -- 172.21.15.28:0/3791591639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282c6780 msgr2=0x5588282c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282c6780 0x5588282c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 -- 172.21.15.28:0/3791591639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588281a93a0 con 0x5588282b75d0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800bcd0 tx=0x7f9ea80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2455d29eeb96a5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9eb9549640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 0x7f9ea40a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009370 tx=0x7f9eac004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9e9effd640 1 -- 172.21.15.28:0/3791591639 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8005220 con 0x5588282b75d0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9e9effd640 1 -- 172.21.15.28:0/3791591639 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8004c80 con 0x5588282b75d0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.346+0000 7f9e9effd640 1 -- 172.21.15.28:0/3791591639 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80117a0 con 0x5588282b75d0 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 -- 172.21.15.28:0/3791591639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 msgr2=0x5588282b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800bcd0 tx=0x7f9ea80107a0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 -- 172.21.15.28:0/3791591639 shutdown_connections 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282c6780 0x5588282c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b75d0 0x5588282b7a10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 --2- 172.21.15.28:0/3791591639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 -- 172.21.15.28:0/3791591639 >> 172.21.15.28:0/3791591639 conn(0x558828280310 msgr2=0x5588282b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 -- 172.21.15.28:0/3791591639 shutdown_connections 2024-09-19T09:34:57.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.347+0000 7f9ebd385740 1 -- 172.21.15.28:0/3791591639 wait complete. 2024-09-19T09:34:57.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 -- start start 2024-09-19T09:34:57.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 Processor -- start 2024-09-19T09:34:57.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 0x5588282c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5588282c9480 con 0x5588282c6780 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5588281d8060 con 0x5588282b8b40 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588282c9600 con 0x5588282b75d0 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9ebd385740 10 client.0 _ll_register_callbacks cb 0x558827f2b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9eb8d48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9eb9549640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9eb8d48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45440/0 (socket says 172.21.15.28:45440) 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9eb8d48640 1 -- 172.21.15.28:0/2042955421 learned_addr learned my addr 172.21.15.28:0/2042955421 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.348+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 0x5588282c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 -- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 msgr2=0x5588282c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 0x5588282c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 -- 172.21.15.28:0/2042955421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 msgr2=0x5588282c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 -- 172.21.15.28:0/2042955421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558827f8e3c0 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 0x5588282c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011010 tx=0x7f9ea80048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e6c3a01ed225aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9e93fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8004ac0 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9ea8005040 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea8004c80 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ea802d070 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9ebd385740 10 client.25723 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.349+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588282d4f20 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8004c80 con 0x5588282b75d0 2024-09-19T09:34:57.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f9ea8026470 con 0x5588282b75d0 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 10 client.25723 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 10 client.25723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 20 client.25723 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588282d6320 0x5588282d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 4 client.25723 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55882822a860 con 0x5588282d6320 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588282d6320 0x5588282d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9e93fff640 10 client.25723 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.350+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588282d6320 0x5588282d8710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6e7b8947272213b server_cookie=dc637b559ce21b6a in_seq=0 out_seq=0 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9e93fff640 1 -- 172.21.15.28:0/2042955421 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9ea00ade80 con 0x5588282d6320 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9e93fff640 10 client.25723 handle_command_reply: tid=0 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 client.25723 shutdown 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 msgr2=0x5588282c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011010 tx=0x7f9ea80048d0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 shutdown_connections 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588282d6320 0x5588282d8710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b8b40 0x5588282c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9e93fff640 0 client.25723 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 --2- 172.21.15.28:0/2042955421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b75d0 0x5588282c6170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.351+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 >> 172.21.15.28:0/2042955421 conn(0x558828280310 msgr2=0x5588282b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.352+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 shutdown_connections 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.352+0000 7f9ebd385740 1 -- 172.21.15.28:0/2042955421 wait complete. 2024-09-19T09:34:57.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.352+0000 7f9ebd385740 20 client.25723 trim_cache size 0 max 16384 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 Processor -- start 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 -- start start 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ab2d0 con 0x5588282c6780 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41aced0 con 0x5588282b75d0 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9ebd385740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ab9a0 con 0x5588282b8b40 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb9549640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb9549640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43126/0 (socket says 172.21.15.28:43126) 2024-09-19T09:34:57.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb9549640 1 -- 172.21.15.28:0/140927130 learned_addr learned my addr 172.21.15.28:0/140927130 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:57.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 -- 172.21.15.28:0/140927130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 msgr2=0x5588282db730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282db730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 -- 172.21.15.28:0/140927130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 msgr2=0x5588282c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb3fff640 1 -- 172.21.15.28:0/140927130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588282b4960 con 0x5588282b8b40 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.365+0000 7f9eb9549640 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282dbc70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5588282b8070 tx=0x7f9ea0049dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76fbfcaa57240b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9e9e7fc640 1 -- 172.21.15.28:0/140927130 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0049040 con 0x5588282b8b40 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9e9e7fc640 1 -- 172.21.15.28:0/140927130 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55882822a860 con 0x5588282b8b40 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9e9e7fc640 1 -- 172.21.15.28:0/140927130 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0049e40 con 0x5588282b8b40 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 -- 172.21.15.28:0/140927130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 msgr2=0x5588282dbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282dbc70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5588282b8070 tx=0x7f9ea0049dd0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 -- 172.21.15.28:0/140927130 shutdown_connections 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282c6780 0x5588282c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282b8b40 0x5588282dbc70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 --2- 172.21.15.28:0/140927130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282db730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 -- 172.21.15.28:0/140927130 >> 172.21.15.28:0/140927130 conn(0x558828280310 msgr2=0x5588282c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 -- 172.21.15.28:0/140927130 shutdown_connections 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.366+0000 7f9ebd385740 1 -- 172.21.15.28:0/140927130 wait complete. 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 -- start start 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 Processor -- start 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 0x5588282d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ac740 con 0x5588282b8b40 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ac070 con 0x5588282b75d0 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41a92d0 con 0x5588282c6780 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9ebd385740 10 client.0 _ll_register_callbacks cb 0x7f9eac009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9eb3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9eb3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43130/0 (socket says 172.21.15.28:43130) 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9eb9549640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 0x5588282d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1388997037 learned_addr learned my addr 172.21.15.28:0/1388997037 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.367+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1388997037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 msgr2=0x5588282d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 0x5588282d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 msgr2=0x5588282dc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:57.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282dc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 -- 172.21.15.28:0/1388997037 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55882833fc60 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb9549640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 0x5588282d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9eb3fff640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00c4df0 tx=0x7f9ea0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2437976aed6678 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9e93fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00d9070 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9ea00b8d20 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55882822a860 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0049040 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9ebd385740 10 client.16173 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558828340e40 con 0x5588282b8b40 2024-09-19T09:34:57.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.368+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55882822a860 con 0x5588282b8b40 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f9ea00f20b0 con 0x5588282b8b40 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 10 client.16173 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 20 client.16173 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588283415b0 0x5588283439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 4 client.16173 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558828224d70 con 0x5588283415b0 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588283415b0 0x5588283439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9e93fff640 10 client.16173 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9eb8d48640 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588283415b0 0x5588283439d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf865c1a9dfb4274 server_cookie=dbc8907729ea5afa in_seq=0 out_seq=0 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9e93fff640 1 -- 172.21.15.28:0/1388997037 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558828224d70 con 0x5588283415b0 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.369+0000 7f9e93fff640 10 client.16173 handle_command_reply: tid=0 2024-09-19T09:34:57.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 client.16173 shutdown 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 msgr2=0x5588282d93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00c4df0 tx=0x7f9ea0005bd0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 shutdown_connections 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588282c6780 0x5588282d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5588282b8b40 0x5588282d93c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x5588283415b0 0x5588283439d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 --2- 172.21.15.28:0/1388997037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588282b75d0 0x5588282dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 >> 172.21.15.28:0/1388997037 conn(0x558828280310 msgr2=0x5588282b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9e93fff640 0 client.16173 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 shutdown_connections 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 1 -- 172.21.15.28:0/1388997037 wait complete. 2024-09-19T09:34:57.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.370+0000 7f9ebd385740 20 client.16173 trim_cache size 0 max 16384 2024-09-19T09:34:57.377 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:57.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 msgr2=0x7f9ea40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 0x7f9ea40a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009370 tx=0x7f9eac004ab0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 msgr2=0x7f9eb4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006010 tx=0x7f9ea00042f0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 shutdown_connections 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ea40a3240 0x7f9ea40a5700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9eb4102790 0x7f9eb4198a50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 --2- 172.21.15.28:0/1821994004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 >> 172.21.15.28:0/1821994004 conn(0x7f9eb40fd8c0 msgr2=0x7f9eb40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 shutdown_connections 2024-09-19T09:34:57.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:57.371+0000 7f9ebafac640 1 -- 172.21.15.28:0/1821994004 wait complete. 2024-09-19T09:34:57.972 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:34:58.232 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:34:58.614 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:34:58 smithi028 ceph-mon[30972]: pgmap v407: 97 pgs: 97 active+clean; 47 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-09-19T09:34:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 -- 172.21.15.28:0/1136596735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 msgr2=0x7f783010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 --2- 172.21.15.28:0/1136596735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783010ea90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7828009ca0 tx=0x7f782802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 -- 172.21.15.28:0/1136596735 shutdown_connections 2024-09-19T09:34:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 --2- 172.21.15.28:0/1136596735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830102790 0x7f78301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 --2- 172.21.15.28:0/1136596735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783010ea90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 --2- 172.21.15.28:0/1136596735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830101de0 0x7f78301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.701+0000 7f7836301640 1 -- 172.21.15.28:0/1136596735 >> 172.21.15.28:0/1136596735 conn(0x7f78300fd8c0 msgr2=0x7f78300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 -- 172.21.15.28:0/1136596735 shutdown_connections 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 -- 172.21.15.28:0/1136596735 wait complete. 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 Processor -- start 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 -- start start 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830101de0 0x7f7830198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830102790 0x7f7830198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.702+0000 7f7836301640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f7836301640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7830111f70 con 0x7f7830106f10 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f7836301640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7830111df0 con 0x7f7830101de0 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f7836301640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78301120f0 con 0x7f7830102790 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43148/0 (socket says 172.21.15.28:43148) 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830102790 0x7f7830198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 -- 172.21.15.28:0/1056791546 learned_addr learned my addr 172.21.15.28:0/1056791546 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f782ffff640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830101de0 0x7f7830198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 -- 172.21.15.28:0/1056791546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830102790 msgr2=0x7f7830198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830102790 0x7f7830198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 -- 172.21.15.28:0/1056791546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830101de0 msgr2=0x7f7830198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830101de0 0x7f7830198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 -- 172.21.15.28:0/1056791546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783019d2e0 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f783489e640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7828031a20 tx=0x7f7828031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f782d7fa640 1 -- 172.21.15.28:0/1056791546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828003b70 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f782d7fa640 1 -- 172.21.15.28:0/1056791546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7828003d10 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783019d570 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.703+0000 7f782d7fa640 1 -- 172.21.15.28:0/1056791546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782802fc80 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.704+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78301a4eb0 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.706+0000 7f782d7fa640 1 -- 172.21.15.28:0/1056791546 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f78280046a0 con 0x7f7830106f10 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.706+0000 7f78386da740 1 Processor -- start 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.706+0000 7f78386da740 1 -- start start 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x560567194a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56056719f370 0x5605671a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5605670fa2f0 con 0x560567195b40 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5605670a1cf0 con 0x5605671945d0 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f78386da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605670f62b0 con 0x56056719f370 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43164/0 (socket says 172.21.15.28:43164) 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 -- 172.21.15.28:0/2757537997 learned_addr learned my addr 172.21.15.28:0/2757537997 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782ffff640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x560567194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56056719f370 0x5605671a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782d7fa640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 0x7f78180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f783489e640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 0x7f78180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 -- 172.21.15.28:0/2757537997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56056719f370 msgr2=0x5605671a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56056719f370 0x5605671a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 -- 172.21.15.28:0/2757537997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 msgr2=0x560567194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x560567194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782f7fe640 1 -- 172.21.15.28:0/2757537997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605670863a0 con 0x560567195b40 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782ffff640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x560567194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.707+0000 7f782d7fa640 1 -- 172.21.15.28:0/1056791546 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f78280f0be0 con 0x7f7830106f10 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78240148e0 tx=0x7f782400a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2af8226b66c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f78167fc640 1 -- 172.21.15.28:0/2757537997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240142d0 con 0x560567195b40 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f78167fc640 1 -- 172.21.15.28:0/2757537997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782400fe50 con 0x560567195b40 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f78167fc640 1 -- 172.21.15.28:0/2757537997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240049c0 con 0x560567195b40 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f78386da740 1 -- 172.21.15.28:0/2757537997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 msgr2=0x560567194090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f78386da740 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78240148e0 tx=0x7f782400a6d0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.708+0000 7f783489e640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 0x7f78180a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78280b9720 tx=0x7f78280880f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- 172.21.15.28:0/2757537997 shutdown_connections 2024-09-19T09:34:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x560567194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56056719f370 0x5605671a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- 172.21.15.28:0/2757537997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560567195b40 0x560567194090 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- 172.21.15.28:0/2757537997 >> 172.21.15.28:0/2757537997 conn(0x56056715d310 msgr2=0x560567190c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- 172.21.15.28:0/2757537997 shutdown_connections 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- 172.21.15.28:0/2757537997 wait complete. 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- start start 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 Processor -- start 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x56056719ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x56056719bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5605671a2070 con 0x56056719f370 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5605670b5060 con 0x5605671945d0 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605671a21f0 con 0x560567195b40 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.709+0000 7f78386da740 10 client.0 _ll_register_callbacks cb 0x560566e088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x56056719bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43176/0 (socket says 172.21.15.28:43176) 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 -- 172.21.15.28:0/2850934290 learned_addr learned my addr 172.21.15.28:0/2850934290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x56056719ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 -- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 msgr2=0x56056719bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x56056719bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 -- 172.21.15.28:0/2850934290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 msgr2=0x56056719ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x56056719ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 -- 172.21.15.28:0/2850934290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560566e6b3c0 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f782f7fe640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7824005760 tx=0x7f7824005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0192c6b2956dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f780f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782400f970 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f782400fb10 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782400fe50 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7824035050 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f78386da740 10 client.16185 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.710+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605671adaa0 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782400fe50 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f7824012450 con 0x56056719f370 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:58.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 20 client.16185 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605671aeea0 0x5605671b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 4 client.16185 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560567107860 con 0x5605671aeea0 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f782ffff640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605671aeea0 0x5605671b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f782ffff640 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605671aeea0 0x5605671b1290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7028ff40e679c715 server_cookie=54962bbf4bcd0138 in_seq=0 out_seq=0 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.711+0000 7f780f7fe640 10 client.16185 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.716+0000 7f780f7fe640 1 -- 172.21.15.28:0/2850934290 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560567107860 con 0x5605671aeea0 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.716+0000 7f780f7fe640 10 client.16185 handle_command_reply: tid=0 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.716+0000 7f78386da740 1 client.16185 shutdown 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.716+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 msgr2=0x56056719c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.716+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7824005760 tx=0x7f7824005580 comp rx=0 tx=0).stop 2024-09-19T09:34:58.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 shutdown_connections 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605671aeea0 0x5605671b1290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719c530 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x56056719bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 --2- 172.21.15.28:0/2850934290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5605671945d0 0x56056719ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 >> 172.21.15.28:0/2850934290 conn(0x56056715d310 msgr2=0x560567196930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f780f7fe640 0 client.16185 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 shutdown_connections 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 1 -- 172.21.15.28:0/2850934290 wait complete. 2024-09-19T09:34:58.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.717+0000 7f78386da740 20 client.16185 trim_cache size 0 max 16384 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 Processor -- start 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 -- start start 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605671945d0 0x5605671ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560567195b40 0x56056719a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ad3e0 con 0x56056719f370 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78301aefe0 con 0x560567195b40 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f78386da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78301adab0 con 0x5605671945d0 2024-09-19T09:34:58.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43192/0 (socket says 172.21.15.28:43192) 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 -- 172.21.15.28:0/946309611 learned_addr learned my addr 172.21.15.28:0/946309611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605671945d0 0x5605671ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782ffff640 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560567195b40 0x56056719a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 -- 172.21.15.28:0/946309611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605671945d0 msgr2=0x5605671ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605671945d0 0x5605671ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 -- 172.21.15.28:0/946309611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560567195b40 msgr2=0x56056719a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560567195b40 0x56056719a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 -- 172.21.15.28:0/946309611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560567197120 con 0x56056719f370 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.731+0000 7f782f7fe640 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f782400aa80 tx=0x7f782400ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7018cdcd187d1f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f7815ffb640 1 -- 172.21.15.28:0/946309611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782400a420 con 0x56056719f370 2024-09-19T09:34:58.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f7815ffb640 1 -- 172.21.15.28:0/946309611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7824005d60 con 0x56056719f370 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f7815ffb640 1 -- 172.21.15.28:0/946309611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240048d0 con 0x56056719f370 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 -- 172.21.15.28:0/946309611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 msgr2=0x56056719adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f782400aa80 tx=0x7f782400ab30 comp rx=0 tx=0).stop 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 -- 172.21.15.28:0/946309611 shutdown_connections 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560567195b40 0x56056719a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56056719f370 0x56056719adc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 --2- 172.21.15.28:0/946309611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605671945d0 0x5605671ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 -- 172.21.15.28:0/946309611 >> 172.21.15.28:0/946309611 conn(0x56056715d310 msgr2=0x5605671b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 -- 172.21.15.28:0/946309611 shutdown_connections 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.732+0000 7f78386da740 1 -- 172.21.15.28:0/946309611 wait complete. 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 -- start start 2024-09-19T09:34:58.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 Processor -- start 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x5605671a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56056719f370 0x560567217a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ae850 con 0x5605671945d0 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ae180 con 0x56056719f370 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78301ab3c0 con 0x560567195b40 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f78386da740 10 client.0 _ll_register_callbacks cb 0x7f782000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f782f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43204/0 (socket says 172.21.15.28:43204) 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f782f7fe640 1 -- 172.21.15.28:0/10095148 learned_addr learned my addr 172.21.15.28:0/10095148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.733+0000 7f782ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x5605671a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 -- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 msgr2=0x5605671a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x5605671a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 -- 172.21.15.28:0/10095148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56056719f370 msgr2=0x560567217a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56056719f370 0x560567217a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 -- 172.21.15.28:0/10095148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560567217f40 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f782f7fe640 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f78240056b0 tx=0x7f782400a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e1d84c72f7fe95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f780f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824012450 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f78240125f0 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7824005d60 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f782400f970 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f78386da740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.734+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560567219220 con 0x5605671945d0 2024-09-19T09:34:58.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7824005d60 con 0x5605671945d0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f782402e450 con 0x5605671945d0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 20 client.16191 populate_metadata read hostname 'smithi028' 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605672197a0 0x56056721bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 4 client.16191 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560567101d70 con 0x5605672197a0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f782ffff640 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605672197a0 0x56056721bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f782ffff640 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605672197a0 0x56056721bbc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8ae1326f62b493 server_cookie=afed9abd70e074ed in_seq=0 out_seq=0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.735+0000 7f780f7fe640 10 client.16191 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f780f7fe640 1 -- 172.21.15.28:0/10095148 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560567101d70 con 0x5605672197a0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f780f7fe640 10 client.16191 handle_command_reply: tid=0 2024-09-19T09:34:58.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 client.16191 shutdown 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 msgr2=0x5605671a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f78240056b0 tx=0x7f782400a780 comp rx=0 tx=0).stop 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 shutdown_connections 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56056719f370 0x560567217a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5605672197a0 0x56056721bbc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560567195b40 0x5605671a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 --2- 172.21.15.28:0/10095148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5605671945d0 0x5605671a29d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f780f7fe640 0 client.16191 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.739+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 >> 172.21.15.28:0/10095148 conn(0x56056715d310 msgr2=0x56056719a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 shutdown_connections 2024-09-19T09:34:58.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f78386da740 1 -- 172.21.15.28:0/10095148 wait complete. 2024-09-19T09:34:58.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f78386da740 20 client.16191 trim_cache size 0 max 16384 2024-09-19T09:34:58.748 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 msgr2=0x7f78180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 0x7f78180a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78280b9720 tx=0x7f78280880f0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 msgr2=0x7f783019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7828031a20 tx=0x7f7828031a50 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 shutdown_connections 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f78180a32c0 0x7f78180a5780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7830106f10 0x7f783019cc00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7830102790 0x7f7830198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 --2- 172.21.15.28:0/1056791546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7830101de0 0x7f7830198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.740+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 >> 172.21.15.28:0/1056791546 conn(0x7f78300fd8c0 msgr2=0x7f78300febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:34:58.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.741+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 shutdown_connections 2024-09-19T09:34:58.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:34:58.741+0000 7f7836301640 1 -- 172.21.15.28:0/1056791546 wait complete. 2024-09-19T09:34:58.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:34:58 smithi078 ceph-mon[35619]: pgmap v407: 97 pgs: 97 active+clean; 47 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-09-19T09:34:58.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:34:58 smithi038 ceph-mon[35271]: pgmap v407: 97 pgs: 97 active+clean; 47 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-09-19T09:34:59.038 INFO:tasks.workunit.client.0.smithi028.stdout: 34 5448 857 2320 1057 1938 1266 2024-09-19T09:35:00.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:35:00.516 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:35:00.779 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 -- 172.21.15.28:0/2877392334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 msgr2=0x7f09ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 --2- 172.21.15.28:0/2877392334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac101200 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009900 tx=0x7f09a802a480 comp rx=0 tx=0).stop 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 -- 172.21.15.28:0/2877392334 shutdown_connections 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 --2- 172.21.15.28:0/2877392334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 --2- 172.21.15.28:0/2877392334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 0x7f09ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 --2- 172.21.15.28:0/2877392334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac101200 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 -- 172.21.15.28:0/2877392334 >> 172.21.15.28:0/2877392334 conn(0x7f09ac0fc7d0 msgr2=0x7f09ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 -- 172.21.15.28:0/2877392334 shutdown_connections 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.245+0000 7f09b379e640 1 -- 172.21.15.28:0/2877392334 wait complete. 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 Processor -- start 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 -- start start 2024-09-19T09:35:01.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 0x7f09ac198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac110f80 con 0x7f09ac100e20 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac110e00 con 0x7f09ac102ef0 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b379e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac111100 con 0x7f09ac102570 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43208/0 (socket says 172.21.15.28:43208) 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 -- 172.21.15.28:0/3746553992 learned_addr learned my addr 172.21.15.28:0/3746553992 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 0x7f09ac198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 -- 172.21.15.28:0/3746553992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 msgr2=0x7f09ac198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 0x7f09ac198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 -- 172.21.15.28:0/3746553992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 msgr2=0x7f09ac19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b153a640 1 -- 172.21.15.28:0/3746553992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac19d4d0 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.246+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09b153a640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f09a8006010 tx=0x7f09a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09a27fc640 1 -- 172.21.15.28:0/3746553992 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a803f070 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09a27fc640 1 -- 172.21.15.28:0/3746553992 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a8031c30 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac19d760 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09a27fc640 1 -- 172.21.15.28:0/3746553992 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8035030 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.247+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a50a0 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.250+0000 7f09a27fc640 1 -- 172.21.15.28:0/3746553992 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f09a8042070 con 0x7f09ac100e20 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.250+0000 7f09b5b77740 1 Processor -- start 2024-09-19T09:35:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.250+0000 7f09b5b77740 1 -- start start 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de469bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555de45f2280 con 0x555de468ca60 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555de4599b10 con 0x555de468d380 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b5b77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555de45ee240 con 0x555de469c150 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43216/0 (socket says 172.21.15.28:43216) 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 -- 172.21.15.28:0/3312263154 learned_addr learned my addr 172.21.15.28:0/3312263154 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b153a640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de469bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 -- 172.21.15.28:0/3312263154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 msgr2=0x555de469e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 -- 172.21.15.28:0/3312263154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 msgr2=0x555de469bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de469bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 -- 172.21.15.28:0/3312263154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de45a61f0 con 0x555de468ca60 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09a27fc640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 0x7f09980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09a40117e0 tx=0x7f09a4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487579cf0f4b73c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09b153a640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 0x7f09980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09a27fc640 1 -- 172.21.15.28:0/3746553992 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f09a80eff40 con 0x7f09ac100e20 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.251+0000 7f09977fe640 1 -- 172.21.15.28:0/3312263154 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4011310 con 0x555de468ca60 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09977fe640 1 -- 172.21.15.28:0/3312263154 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a4003e50 con 0x555de468ca60 2024-09-19T09:35:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09977fe640 1 -- 172.21.15.28:0/3312263154 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004590 con 0x555de468ca60 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 -- 172.21.15.28:0/3312263154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 msgr2=0x555de468ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b153a640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 0x7f09980a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09a80edde0 tx=0x7f09a80f3a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09a40117e0 tx=0x7f09a4011ca0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 -- 172.21.15.28:0/3312263154 shutdown_connections 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de469bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3312263154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de468ce40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 -- 172.21.15.28:0/3312263154 >> 172.21.15.28:0/3312263154 conn(0x555de46890f0 msgr2=0x555de46894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 -- 172.21.15.28:0/3312263154 shutdown_connections 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.252+0000 7f09b5b77740 1 -- 172.21.15.28:0/3312263154 wait complete. 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 -- start start 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 Processor -- start 2024-09-19T09:35:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de4699b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555de469ed80 con 0x555de468d380 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555de45acf10 con 0x555de469c150 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555de469ef00 con 0x555de468ca60 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b5b77740 10 client.0 _ll_register_callbacks cb 0x555de4575a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b0d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b0d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43220/0 (socket says 172.21.15.28:43220) 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45556/0 (socket says 172.21.15.28:45556) 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.253+0000 7f09b0d39640 1 -- 172.21.15.28:0/727625721 learned_addr learned my addr 172.21.15.28:0/727625721 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de4699b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 -- 172.21.15.28:0/727625721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 msgr2=0x555de4699b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de4699b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 -- 172.21.15.28:0/727625721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 msgr2=0x555de4699630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 -- 172.21.15.28:0/727625721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de43636a0 con 0x555de468ca60 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b0d39640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09a400fdc0 tx=0x7f09a4004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b7d84d68d68ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f0994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4025040 con 0x555de468ca60 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a4003e50 con 0x555de468ca60 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f09a4011310 con 0x555de468ca60 2024-09-19T09:35:01.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09a4004a40 con 0x555de468ca60 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b5b77740 10 client.25759 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.254+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555de46aaa80 con 0x555de468ca60 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a4003e50 con 0x555de468ca60 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f09a4018020 con 0x555de468ca60 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 20 client.25759 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de46abdd0 0x555de46ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 4 client.25759 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555de45ff7f0 con 0x555de46abdd0 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b0d39640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de46abdd0 0x555de46ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.255+0000 7f09b0d39640 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de46abdd0 0x555de46ae1c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22ac64183d42b17 server_cookie=39a3575abeeb8fba in_seq=0 out_seq=0 2024-09-19T09:35:01.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.256+0000 7f0994ff9640 10 client.25759 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.259+0000 7f0994ff9640 1 -- 172.21.15.28:0/727625721 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555de45ff7f0 con 0x555de46abdd0 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.259+0000 7f0994ff9640 10 client.25759 handle_command_reply: tid=0 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.259+0000 7f09b5b77740 1 client.25759 shutdown 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 msgr2=0x555de46990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09a400fdc0 tx=0x7f09a4004170 comp rx=0 tx=0).stop 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 shutdown_connections 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de4699b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de46abdd0 0x555de46ae1c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de4699630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 --2- 172.21.15.28:0/727625721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de46990f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f0994ff9640 0 client.25759 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 >> 172.21.15.28:0/727625721 conn(0x555de46890f0 msgr2=0x555de468ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 shutdown_connections 2024-09-19T09:35:01.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 1 -- 172.21.15.28:0/727625721 wait complete. 2024-09-19T09:35:01.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.260+0000 7f09b5b77740 20 client.25759 trim_cache size 0 max 16384 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.277+0000 7f09b5b77740 1 Processor -- start 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.277+0000 7f09b5b77740 1 -- start start 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de468eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de46b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ab310 con 0x555de468d380 2024-09-19T09:35:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1acf10 con 0x555de469c150 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b5b77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ab9e0 con 0x555de468ca60 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43226/0 (socket says 172.21.15.28:43226) 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 -- 172.21.15.28:0/3908348136 learned_addr learned my addr 172.21.15.28:0/3908348136 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de468eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de46b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 -- 172.21.15.28:0/3908348136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 msgr2=0x555de468eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de468eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 -- 172.21.15.28:0/3908348136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 msgr2=0x555de46b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de46b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 -- 172.21.15.28:0/3908348136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de46980a0 con 0x555de468d380 2024-09-19T09:35:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de46b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.278+0000 7f09b0d39640 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fa0 tx=0x7f099c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b749e154c9ac6ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f0996ffd640 1 -- 172.21.15.28:0/3908348136 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c004310 con 0x555de468d380 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f0996ffd640 1 -- 172.21.15.28:0/3908348136 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c0044b0 con 0x555de468d380 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f0996ffd640 1 -- 172.21.15.28:0/3908348136 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c010a30 con 0x555de468d380 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 -- 172.21.15.28:0/3908348136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 msgr2=0x555de46b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fa0 tx=0x7f099c005f90 comp rx=0 tx=0).stop 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 -- 172.21.15.28:0/3908348136 shutdown_connections 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de469c150 0x555de46b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468d380 0x555de46b39f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 --2- 172.21.15.28:0/3908348136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de468ca60 0x555de468eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 -- 172.21.15.28:0/3908348136 >> 172.21.15.28:0/3908348136 conn(0x555de46890f0 msgr2=0x555de468e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 -- 172.21.15.28:0/3908348136 shutdown_connections 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.279+0000 7f09b5b77740 1 -- 172.21.15.28:0/3908348136 wait complete. 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 -- start start 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 Processor -- start 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de46a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ac780 con 0x555de468ca60 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ac0b0 con 0x555de468d380 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1a9310 con 0x555de469c150 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b5b77740 10 client.0 _ll_register_callbacks cb 0x7f09a80f38f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:01.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.280+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43242/0 (socket says 172.21.15.28:43242) 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 -- 172.21.15.28:0/1090481388 learned_addr learned my addr 172.21.15.28:0/1090481388 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b0d39640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de46a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 -- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 msgr2=0x555de469fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 -- 172.21.15.28:0/1090481388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 msgr2=0x555de46a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de46a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 -- 172.21.15.28:0/1090481388 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de46a02d0 con 0x555de468ca60 2024-09-19T09:35:01.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b1d3b640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009560 tx=0x7f09a4004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982510ba7248ad04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f0994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004e50 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f09a4011110 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c0044b0 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09a4011310 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b5b77740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.281+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555de4716680 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c0044b0 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f09a4018040 con 0x555de468ca60 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 20 client.16209 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de4716da0 0x555de47191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 4 client.16209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555de45f9d00 con 0x555de4716da0 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b0d39640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de4716da0 0x555de47191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f09b0d39640 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de4716da0 0x555de47191c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5d1922e0c93e4 server_cookie=787813d9c4e00ed5 in_seq=0 out_seq=0 2024-09-19T09:35:01.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.282+0000 7f0994ff9640 10 client.16209 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:01 smithi028 ceph-mon[30972]: pgmap v408: 97 pgs: 97 active+clean; 51 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T09:35:01.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f0994ff9640 1 -- 172.21.15.28:0/1090481388 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555de45f9d00 con 0x555de4716da0 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f0994ff9640 10 client.16209 handle_command_reply: tid=0 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 client.16209 shutdown 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 msgr2=0x555de46a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009560 tx=0x7f09a4004170 comp rx=0 tx=0).stop 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 shutdown_connections 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555de468d380 0x555de46a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555de4716da0 0x555de47191c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555de469c150 0x555de469fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f0994ff9640 0 client.16209 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 --2- 172.21.15.28:0/1090481388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555de468ca60 0x555de46a4550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 >> 172.21.15.28:0/1090481388 conn(0x555de46890f0 msgr2=0x555de4688270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 shutdown_connections 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.538+0000 7f09b5b77740 1 -- 172.21.15.28:0/1090481388 wait complete. 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b5b77740 20 client.16209 trim_cache size 0 max 16384 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "d6c3b6be-3fe0-4496-a752-e617b79d6306", 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2024-09-19T09:35:01.541 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 msgr2=0x7f09980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 0x7f09980a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09a80edde0 tx=0x7f09a80f3a20 comp rx=0 tx=0).stop 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 msgr2=0x7f09ac198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f09a8006010 tx=0x7f09a8003c10 comp rx=0 tx=0).stop 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 shutdown_connections 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f09980a3240 0x7f09980a5700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09ac102ef0 0x7f09ac19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09ac102570 0x7f09ac198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 --2- 172.21.15.28:0/3746553992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09ac100e20 0x7f09ac198520 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 >> 172.21.15.28:0/3746553992 conn(0x7f09ac0fc7d0 msgr2=0x7f09ac06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 shutdown_connections 2024-09-19T09:35:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:01.539+0000 7f09b379e640 1 -- 172.21.15.28:0/3746553992 wait complete. 2024-09-19T09:35:01.558 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:01 smithi078 ceph-mon[35619]: pgmap v408: 97 pgs: 97 active+clean; 51 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T09:35:01.627 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:01 smithi038 ceph-mon[35271]: pgmap v408: 97 pgs: 97 active+clean; 51 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-19T09:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 -- 172.21.15.28:0/674951692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918101b70 msgr2=0x7f3918101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 --2- 172.21.15.28:0/674951692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918101b70 0x7f3918101f70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3914009860 tx=0x7f391402a340 comp rx=0 tx=0).stop 2024-09-19T09:35:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 -- 172.21.15.28:0/674951692 shutdown_connections 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 --2- 172.21.15.28:0/674951692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 0x7f3918109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 --2- 172.21.15.28:0/674951692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918102d60 0x7f39181031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 --2- 172.21.15.28:0/674951692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918101b70 0x7f3918101f70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 -- 172.21.15.28:0/674951692 >> 172.21.15.28:0/674951692 conn(0x7f39180fd2f0 msgr2=0x7f39180ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 -- 172.21.15.28:0/674951692 shutdown_connections 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.749+0000 7f392048b640 1 -- 172.21.15.28:0/674951692 wait complete. 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 Processor -- start 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 -- start start 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918101b70 0x7f3918197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 0x7f391819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d7d0 con 0x7f3918102d60 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d650 con 0x7f3918103720 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f392048b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d950 con 0x7f3918101b70 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f391da26640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f391da26640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43262/0 (socket says 172.21.15.28:43262) 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f391ea28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 0x7f391819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f391da26640 1 -- 172.21.15.28:0/4078704816 learned_addr learned my addr 172.21.15.28:0/4078704816 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.750+0000 7f391e227640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918101b70 0x7f3918197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 -- 172.21.15.28:0/4078704816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918101b70 msgr2=0x7f3918197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918101b70 0x7f3918197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 -- 172.21.15.28:0/4078704816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 msgr2=0x7f391819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 0x7f391819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 -- 172.21.15.28:0/4078704816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391819fad0 con 0x7f3918102d60 2024-09-19T09:35:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f391da26640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f390800e9c0 tx=0x7f390800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f390f7fe640 1 -- 172.21.15.28:0/4078704816 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800cde0 con 0x7f3918102d60 2024-09-19T09:35:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f390f7fe640 1 -- 172.21.15.28:0/4078704816 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39080035f0 con 0x7f3918102d60 2024-09-19T09:35:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391819fdc0 con 0x7f3918102d60 2024-09-19T09:35:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f390f7fe640 1 -- 172.21.15.28:0/4078704816 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908010690 con 0x7f3918102d60 2024-09-19T09:35:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.751+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39181a0280 con 0x7f3918102d60 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.754+0000 7f390f7fe640 1 -- 172.21.15.28:0/4078704816 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f39080037a0 con 0x7f3918102d60 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.754+0000 7f3923224740 1 Processor -- start 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.754+0000 7f3923224740 1 -- start start 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.754+0000 7f3923224740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.754+0000 7f3923224740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec67da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f3923224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec6a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f3923224740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ac8ebc62f0 con 0x55ac8ec629a0 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f3923224740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac8eb6e270 con 0x55ac8ec632e0 2024-09-19T09:35:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f3923224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac8ebc22b0 con 0x55ac8ec68370 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391e227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec67da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391e227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec67da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:53646/0 (socket says 172.21.15.28:53646) 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391e227640 1 -- 172.21.15.28:0/725423844 learned_addr learned my addr 172.21.15.28:0/725423844 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec6a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f390f7fe640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 0x7f39040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391ea28640 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 -- 172.21.15.28:0/725423844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 msgr2=0x55ac8ec67da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391ea28640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 0x7f39040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec67da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 -- 172.21.15.28:0/725423844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 msgr2=0x55ac8ec62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f391da26640 1 -- 172.21.15.28:0/725423844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8eb6d340 con 0x55ac8ec68370 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.755+0000 7f390f7fe640 1 -- 172.21.15.28:0/4078704816 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f39080cfc10 con 0x7f3918102d60 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.756+0000 7f391da26640 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec6a830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39080cd790 tx=0x7f3908003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e7ffc46af3e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.756+0000 7f390cbfe640 1 -- 172.21.15.28:0/725423844 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908003b10 con 0x55ac8ec68370 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.756+0000 7f390cbfe640 1 -- 172.21.15.28:0/725423844 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390809c960 con 0x55ac8ec68370 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.756+0000 7f390cbfe640 1 -- 172.21.15.28:0/725423844 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908049160 con 0x55ac8ec68370 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- 172.21.15.28:0/725423844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 msgr2=0x55ac8ec6a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f391ea28640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 0x7f39040a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3910005fd0 tx=0x7f3910004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec6a830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39080cd790 tx=0x7f3908003cd0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- 172.21.15.28:0/725423844 shutdown_connections 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec6a830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 --2- 172.21.15.28:0/725423844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- 172.21.15.28:0/725423844 >> 172.21.15.28:0/725423844 conn(0x55ac8ec5e010 msgr2=0x55ac8ec5e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- 172.21.15.28:0/725423844 shutdown_connections 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- 172.21.15.28:0/725423844 wait complete. 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 -- start start 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 Processor -- start 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.757+0000 7f3923224740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ac8ec6b280 con 0x55ac8ec68370 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac8eb4f290 con 0x55ac8ec629a0 2024-09-19T09:35:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac8ec6b400 con 0x55ac8ec632e0 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 10 client.0 _ll_register_callbacks cb 0x55ac8eb80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391ea28640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391da26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:53654/0 (socket says 172.21.15.28:53654) 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 -- 172.21.15.28:0/1707475010 learned_addr learned my addr 172.21.15.28:0/1707475010 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391da26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45626/0 (socket says 172.21.15.28:45626) 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 -- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 msgr2=0x55ac8ec6d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 -- 172.21.15.28:0/1707475010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 msgr2=0x55ac8ec74050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec74050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 -- 172.21.15.28:0/1707475010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8e8d9210 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391da26640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f391e227640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3914009830 tx=0x7f39140306e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7989c710a2b302d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914037040 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3914004240 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.758+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390809c960 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3914004440 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 10 client.25823 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac8e9a5620 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390809c960 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f39140035f0 con 0x55ac8ec629a0 2024-09-19T09:35:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 20 client.25823 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ec78820 0x55ac8ec7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 4 client.25823 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac8ebd3a20 con 0x55ac8ec78820 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.759+0000 7f391da26640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ec78820 0x55ac8ec7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.760+0000 7f3901ffb640 10 client.25823 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.760+0000 7f391da26640 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ec78820 0x55ac8ec7ac70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853b1469be883733 server_cookie=48c769261474283c in_seq=0 out_seq=0 2024-09-19T09:35:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3901ffb640 1 -- 172.21.15.28:0/1707475010 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac8ebd3a20 con 0x55ac8ec78820 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3901ffb640 10 client.25823 handle_command_reply: tid=0 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 client.25823 shutdown 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 msgr2=0x55ac8ec6cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3914009830 tx=0x7f39140306e0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 shutdown_connections 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ec78820 0x55ac8ec7ac70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec74050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 --2- 172.21.15.28:0/1707475010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3901ffb640 0 client.25823 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 >> 172.21.15.28:0/1707475010 conn(0x55ac8ec5e010 msgr2=0x55ac8ec5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 shutdown_connections 2024-09-19T09:35:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 1 -- 172.21.15.28:0/1707475010 wait complete. 2024-09-19T09:35:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.763+0000 7f3923224740 20 client.25823 trim_cache size 0 max 16384 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 Processor -- start 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 -- start start 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec5c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a6d30 con 0x55ac8ec629a0 2024-09-19T09:35:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a8930 con 0x55ac8ec632e0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f3923224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a7400 con 0x55ac8ec68370 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f391e227640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f391e227640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43290/0 (socket says 172.21.15.28:43290) 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f391e227640 1 -- 172.21.15.28:0/1202176249 learned_addr learned my addr 172.21.15.28:0/1202176249 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.778+0000 7f391ea28640 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec5c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391da26640 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 -- 172.21.15.28:0/1202176249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 msgr2=0x55ac8ec5c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec5c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 -- 172.21.15.28:0/1202176249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 msgr2=0x55ac8ec74010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec74010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 -- 172.21.15.28:0/1202176249 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8ec6ace0 con 0x55ac8ec629a0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f391e227640 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3914009860 tx=0x7f3914000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a6c092d8c04f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f3903fff640 1 -- 172.21.15.28:0/1202176249 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914057070 con 0x55ac8ec629a0 2024-09-19T09:35:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f3903fff640 1 -- 172.21.15.28:0/1202176249 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914003c30 con 0x55ac8ec629a0 2024-09-19T09:35:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.779+0000 7f3903fff640 1 -- 172.21.15.28:0/1202176249 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914037030 con 0x55ac8ec629a0 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- 172.21.15.28:0/1202176249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 msgr2=0x55ac8ec6cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3914009860 tx=0x7f3914000bc0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- 172.21.15.28:0/1202176249 shutdown_connections 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec632e0 0x55ac8ec74010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec68370 0x55ac8ec5c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- 172.21.15.28:0/1202176249 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec629a0 0x55ac8ec6cf90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- 172.21.15.28:0/1202176249 >> 172.21.15.28:0/1202176249 conn(0x55ac8ec5e010 msgr2=0x55ac8ec6c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- 172.21.15.28:0/1202176249 shutdown_connections 2024-09-19T09:35:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- 172.21.15.28:0/1202176249 wait complete. 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- start start 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 Processor -- start 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec71570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a81a0 con 0x55ac8ec68370 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a7ad0 con 0x55ac8ec629a0 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39181a4d30 con 0x55ac8ec632e0 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.780+0000 7f3923224740 10 client.0 _ll_register_callbacks cb 0x7f39140096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43294/0 (socket says 172.21.15.28:43294) 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 -- 172.21.15.28:0/1858970590 learned_addr learned my addr 172.21.15.28:0/1858970590 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391e227640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec71570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391da26640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 -- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 msgr2=0x55ac8ec71ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec71ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 -- 172.21.15.28:0/1858970590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 msgr2=0x55ac8ec71570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec71570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 -- 172.21.15.28:0/1858970590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8ec68110 con 0x55ac8ec68370 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f391ea28640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3910012870 tx=0x7f39100096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08b1407b37662f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f3901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910010920 con 0x55ac8ec68370 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f3910010ac0 con 0x55ac8ec68370 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3914003c30 con 0x55ac8ec68370 2024-09-19T09:35:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.781+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3910010cc0 con 0x55ac8ec68370 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac8ec742e0 con 0x55ac8ec68370 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914003c30 con 0x55ac8ec68370 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f391001f580 con 0x55ac8ec68370 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 20 client.16227 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ece3db0 0x55ac8ece61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 4 client.16227 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac8ebcdd70 con 0x55ac8ece3db0 2024-09-19T09:35:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f391e227640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ece3db0 0x55ac8ece61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.783+0000 7f3901ffb640 10 client.16227 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:01.782+0000 7f391e227640 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ece3db0 0x55ac8ece61d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75d18f024b016e4 server_cookie=8e4501216f88d5a9 in_seq=0 out_seq=0 2024-09-19T09:35:02.181 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:35:02.213 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:01 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-09-19T09:35:02.213 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:01 smithi078 systemd[1]: Stopped Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:35:02.213 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:01 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Consumed 2.749s CPU time. 2024-09-19T09:35:02.213 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:01 smithi078 systemd[1]: Starting Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:35:02.214 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:02 smithi078 podman[69049]: 2024-09-19 09:35:02.005895694 +0000 UTC m=+0.018370311 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:35:02.449 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:02.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:02 smithi028 ceph-mon[30972]: scrub queued for path: / 2024-09-19T09:35:02.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:02 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:02.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:02 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:02 smithi078 ceph-mon[35619]: scrub queued for path: / 2024-09-19T09:35:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:02 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:02.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:02 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:02.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:02 smithi038 ceph-mon[35271]: scrub queued for path: / 2024-09-19T09:35:02.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:02 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:02.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:02 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:03.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.012+0000 7f14827fc640 1 -- 172.21.15.28:0/2514216685 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14740319b0 con 0x7f148410e580 2024-09-19T09:35:03.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.013+0000 7f1489d09640 1 -- 172.21.15.28:0/2514216685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410e580 msgr2=0x7f148410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.013+0000 7f1489d09640 1 --2- 172.21.15.28:0/2514216685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410e580 0x7f148410e960 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1474009930 tx=0x7f147402a440 comp rx=0 tx=0).stop 2024-09-19T09:35:03.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.014+0000 7f1489d09640 1 -- 172.21.15.28:0/2514216685 shutdown_connections 2024-09-19T09:35:03.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.014+0000 7f1489d09640 1 --2- 172.21.15.28:0/2514216685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410ef30 0x7f148410d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.014+0000 7f1489d09640 1 --2- 172.21.15.28:0/2514216685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410d9b0 0x7f1484115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.014+0000 7f1489d09640 1 --2- 172.21.15.28:0/2514216685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410e580 0x7f148410e960 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.014+0000 7f1489d09640 1 -- 172.21.15.28:0/2514216685 >> 172.21.15.28:0/2514216685 conn(0x7f148406e140 msgr2=0x7f148406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.018+0000 7f1489d09640 1 -- 172.21.15.28:0/2514216685 shutdown_connections 2024-09-19T09:35:03.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.019+0000 7f1489d09640 1 -- 172.21.15.28:0/2514216685 wait complete. 2024-09-19T09:35:03.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.019+0000 7f1489d09640 1 Processor -- start 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 -- start start 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410d9b0 0x7f148419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410ef30 0x7f14841a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1484118a70 con 0x7f148410ef30 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14841188f0 con 0x7f148410d9b0 2024-09-19T09:35:03.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1489d09640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1484118bf0 con 0x7f148410e580 2024-09-19T09:35:03.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45682/0 (socket says 172.21.15.28:45682) 2024-09-19T09:35:03.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.020+0000 7f1482ffd640 1 -- 172.21.15.28:0/1774557479 learned_addr learned my addr 172.21.15.28:0/1774557479 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.021+0000 7f14837fe640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410d9b0 0x7f148419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 -- 172.21.15.28:0/1774557479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410d9b0 msgr2=0x7f148419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410d9b0 0x7f148419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 -- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410ef30 msgr2=0x7f14841a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410ef30 0x7f14841a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 -- 172.21.15.28:0/1774557479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14841a4310 con 0x7f148410e580 2024-09-19T09:35:03.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.022+0000 7f1482ffd640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f146c00df40 tx=0x7f146c008740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.024+0000 7f1480ff9640 1 -- 172.21.15.28:0/1774557479 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c018070 con 0x7f148410e580 2024-09-19T09:35:03.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.024+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a4600 con 0x7f148410e580 2024-09-19T09:35:03.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.024+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1484119230 con 0x7f148410e580 2024-09-19T09:35:03.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.024+0000 7f1480ff9640 1 -- 172.21.15.28:0/1774557479 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f146c008e50 con 0x7f148410e580 2024-09-19T09:35:03.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.025+0000 7f1480ff9640 1 -- 172.21.15.28:0/1774557479 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c01b720 con 0x7f148410e580 2024-09-19T09:35:03.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.029+0000 7f1480ff9640 1 -- 172.21.15.28:0/1774557479 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f146c01b070 con 0x7f148410e580 2024-09-19T09:35:03.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 Processor -- start 2024-09-19T09:35:03.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 -- start start 2024-09-19T09:35:03.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc1e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc1ece0 0x5558dcc31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 0x5558dcc34280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5558dcb842e0 con 0x5558dcc1ece0 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5558dcb2bce0 con 0x5558dcc31e90 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.030+0000 7f148c0e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5558dcb802a0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.031+0000 7f1483fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc1e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.031+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 0x5558dcc34280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.031+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 0x5558dcc34280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:53700/0 (socket says 172.21.15.28:53700) 2024-09-19T09:35:03.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.031+0000 7f1482ffd640 1 -- 172.21.15.28:0/4171881975 learned_addr learned my addr 172.21.15.28:0/4171881975 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:03.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.032+0000 7f1480ff9640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 0x7f14700a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.032+0000 7f14837fe640 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc1ece0 0x5558dcc31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.032+0000 7f1480ff9640 1 -- 172.21.15.28:0/1774557479 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f146c0d2c40 con 0x7f148410e580 2024-09-19T09:35:03.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 0x7f14700a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 -- 172.21.15.28:0/4171881975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 msgr2=0x5558dcc34280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 0x5558dcc34280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 -- 172.21.15.28:0/4171881975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc1ece0 msgr2=0x5558dcc31950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc1ece0 0x5558dcc31950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.033+0000 7f1483fff640 1 -- 172.21.15.28:0/4171881975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558dcb10440 con 0x5558dcc1e3c0 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.034+0000 7f1483fff640 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc1e7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f147c013530 tx=0x7f147c013930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=290040dcfb44843d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.034+0000 7f1483fff640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 0x7f14700a5890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f147c013e30 tx=0x7f147c019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.036+0000 7f1479ffb640 1 -- 172.21.15.28:0/4171881975 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0035f0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.036+0000 7f1479ffb640 1 -- 172.21.15.28:0/4171881975 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c003c70 con 0x5558dcc1e3c0 2024-09-19T09:35:03.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.036+0000 7f1479ffb640 1 -- 172.21.15.28:0/4171881975 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c01b900 con 0x5558dcc1e3c0 2024-09-19T09:35:03.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.037+0000 7f148c0e2740 1 -- 172.21.15.28:0/4171881975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 msgr2=0x5558dcc1e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.037+0000 7f148c0e2740 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc1e7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f147c013530 tx=0x7f147c013930 comp rx=0 tx=0).stop 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 -- 172.21.15.28:0/4171881975 shutdown_connections 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc31e90 0x5558dcc34280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc1ece0 0x5558dcc31950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 --2- 172.21.15.28:0/4171881975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc1e7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 -- 172.21.15.28:0/4171881975 >> 172.21.15.28:0/4171881975 conn(0x5558dcc1aa30 msgr2=0x5558dcc1ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 -- 172.21.15.28:0/4171881975 shutdown_connections 2024-09-19T09:35:03.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 -- 172.21.15.28:0/4171881975 wait complete. 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.038+0000 7f148c0e2740 1 -- start start 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 Processor -- start 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc2ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc2efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5558dcc349e0 con 0x5558dcc31e90 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5558dcb3f050 con 0x5558dcc1ece0 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5558dcc34b60 con 0x5558dcc1e3c0 2024-09-19T09:35:03.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.039+0000 7f148c0e2740 10 client.0 _ll_register_callbacks cb 0x5558dcb7f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:03.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f1483fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc2efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f1483fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc2efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43346/0 (socket says 172.21.15.28:43346) 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f1483fff640 1 -- 172.21.15.28:0/3800906054 learned_addr learned my addr 172.21.15.28:0/3800906054 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f1482ffd640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc2ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 -- 172.21.15.28:0/3800906054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 msgr2=0x5558dcc2ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc2ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 -- 172.21.15.28:0/3800906054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 msgr2=0x5558dcc2efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc2efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.041+0000 7f14837fe640 1 -- 172.21.15.28:0/3800906054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558dc8f53c0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.044+0000 7f14837fe640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc31760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1474005ab0 tx=0x7f14740045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe880eaf32e49c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.044+0000 7f1462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:03.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.047+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474035040 con 0x5558dcc1e3c0 2024-09-19T09:35:03.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.047+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1474005550 con 0x5558dcc1e3c0 2024-09-19T09:35:03.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.049+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c003c70 con 0x5558dcc1e3c0 2024-09-19T09:35:03.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.050+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14740046c0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.051+0000 7f148c0e2740 10 client.25792 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:03.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.051+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558dcc40430 con 0x5558dcc1e3c0 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c003c70 con 0x5558dcc1e3c0 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f1474003c10 con 0x5558dcc1e3c0 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 10 client.25792 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 20 client.25792 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dcc41940 0x5558dcc43d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 4 client.25792 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:03.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.052+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558dcb91850 con 0x5558dcc41940 2024-09-19T09:35:03.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.059+0000 7f1482ffd640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dcc41940 0x5558dcc43d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.063+0000 7f1482ffd640 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dcc41940 0x5558dcc43d60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c46c1fa61ec743 server_cookie=6d93a103b8130e27 in_seq=0 out_seq=0 2024-09-19T09:35:03.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.063+0000 7f1462ffd640 10 client.25792 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:03.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.068+0000 7f1462ffd640 1 -- 172.21.15.28:0/3800906054 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558dcb91850 con 0x5558dcc41940 2024-09-19T09:35:03.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.068+0000 7f1462ffd640 10 client.25792 handle_command_reply: tid=0 2024-09-19T09:35:03.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.068+0000 7f148c0e2740 1 client.25792 shutdown 2024-09-19T09:35:03.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 msgr2=0x5558dcc31760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc31760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1474005ab0 tx=0x7f14740045a0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 shutdown_connections 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc2ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dcc41940 0x5558dcc43d60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc2efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 --2- 172.21.15.28:0/3800906054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc31760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 >> 172.21.15.28:0/3800906054 conn(0x5558dcc1aa30 msgr2=0x5558dcc19b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f1462ffd640 0 client.25792 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 shutdown_connections 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 1 -- 172.21.15.28:0/3800906054 wait complete. 2024-09-19T09:35:03.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.077+0000 7f148c0e2740 20 client.25792 trim_cache size 0 max 16384 2024-09-19T09:35:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.116+0000 7f148c0e2740 1 Processor -- start 2024-09-19T09:35:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.116+0000 7f148c0e2740 1 -- start start 2024-09-19T09:35:03.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f146400e3d0 con 0x5558dcc31e90 2024-09-19T09:35:03.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f146400ffd0 con 0x5558dcc1ece0 2024-09-19T09:35:03.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.117+0000 7f148c0e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f146400eaa0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.120+0000 7f14837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.120+0000 7f14837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45706/0 (socket says 172.21.15.28:45706) 2024-09-19T09:35:03.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.120+0000 7f14837fe640 1 -- 172.21.15.28:0/641896609 learned_addr learned my addr 172.21.15.28:0/641896609 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:03.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.120+0000 7f1482ffd640 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f1483fff640 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f14837fe640 1 -- 172.21.15.28:0/641896609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 msgr2=0x5558dcc49670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f14837fe640 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc49670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f14837fe640 1 -- 172.21.15.28:0/641896609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 msgr2=0x5558dcc49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f14837fe640 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.121+0000 7f14837fe640 1 -- 172.21.15.28:0/641896609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558dcc2cfb0 con 0x5558dcc1e3c0 2024-09-19T09:35:03.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.123+0000 7f14837fe640 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1474031a30 tx=0x7f1474004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef3f8af2acad45b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.125+0000 7f14797fa640 1 -- 172.21.15.28:0/641896609 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474035070 con 0x5558dcc1e3c0 2024-09-19T09:35:03.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.125+0000 7f14797fa640 1 -- 172.21.15.28:0/641896609 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1474005550 con 0x5558dcc1e3c0 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.125+0000 7f14797fa640 1 -- 172.21.15.28:0/641896609 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474003830 con 0x5558dcc1e3c0 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 -- 172.21.15.28:0/641896609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 msgr2=0x5558dcc3dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1474031a30 tx=0x7f1474004610 comp rx=0 tx=0).stop 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 -- 172.21.15.28:0/641896609 shutdown_connections 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc1ece0 0x5558dcc49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 --2- 172.21.15.28:0/641896609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1e3c0 0x5558dcc3dd10 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1474031a30 tx=0x7f1474004610 comp rx=0 tx=0).stop 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.126+0000 7f148c0e2740 1 -- 172.21.15.28:0/641896609 >> 172.21.15.28:0/641896609 conn(0x5558dcc1aa30 msgr2=0x5558dcc19b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.128+0000 7f148c0e2740 1 -- 172.21.15.28:0/641896609 shutdown_connections 2024-09-19T09:35:03.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.128+0000 7f148c0e2740 1 -- 172.21.15.28:0/641896609 wait complete. 2024-09-19T09:35:03.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.128+0000 7f148c0e2740 1 -- start start 2024-09-19T09:35:03.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 Processor -- start 2024-09-19T09:35:03.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc368f0 0x5558dccab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f146400f840 con 0x5558dcc31e90 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f146400f170 con 0x5558dcc368f0 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f146400c3b0 con 0x5558dcc1ece0 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.129+0000 7f148c0e2740 10 client.0 _ll_register_callbacks cb 0x7f1474009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.130+0000 7f14837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.130+0000 7f1483fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc368f0 0x5558dccab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.130+0000 7f14837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45708/0 (socket says 172.21.15.28:45708) 2024-09-19T09:35:03.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.130+0000 7f14837fe640 1 -- 172.21.15.28:0/2655310018 learned_addr learned my addr 172.21.15.28:0/2655310018 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:03.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f1482ffd640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f14837fe640 1 -- 172.21.15.28:0/2655310018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc368f0 msgr2=0x5558dccab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f14837fe640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc368f0 0x5558dccab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f14837fe640 1 -- 172.21.15.28:0/2655310018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 msgr2=0x5558dcc363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:03.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f14837fe640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.131+0000 7f14837fe640 1 -- 172.21.15.28:0/2655310018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558dccabd00 con 0x5558dcc1ece0 2024-09-19T09:35:03.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.139+0000 7f14837fe640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1474031c70 tx=0x7f1474031510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ec082f166c1746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:03.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.139+0000 7f1462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:03.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.140+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474035040 con 0x5558dcc1ece0 2024-09-19T09:35:03.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.140+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f1474003b30 con 0x5558dcc1ece0 2024-09-19T09:35:03.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.141+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1474005550 con 0x5558dcc1ece0 2024-09-19T09:35:03.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.145+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1474003d30 con 0x5558dcc1ece0 2024-09-19T09:35:03.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.146+0000 7f148c0e2740 10 client.25798 fetch_fsmap learned FSMap version 29 2024-09-19T09:35:03.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.146+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558dccacff0 con 0x5558dcc1ece0 2024-09-19T09:35:03.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.149+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1474005550 con 0x5558dcc1ece0 2024-09-19T09:35:03.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.149+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f147405e020 con 0x5558dcc1ece0 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 10 client.25798 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 20 client.25798 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dccad480 0x5558dccaf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 4 client.25798 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:35:03.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.150+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558dcb8bd60 con 0x5558dccad480 2024-09-19T09:35:03.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.151+0000 7f1482ffd640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dccad480 0x5558dccaf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:03.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.155+0000 7f1482ffd640 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dccad480 0x5558dccaf8a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a54c4eafd458997 server_cookie=80697bdf18e33ab9 in_seq=0 out_seq=0 2024-09-19T09:35:03.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.156+0000 7f1462ffd640 10 client.25798 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:03.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.164+0000 7f1462ffd640 1 -- 172.21.15.28:0/2655310018 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5558dcb8bd60 con 0x5558dccad480 2024-09-19T09:35:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.164+0000 7f1462ffd640 10 client.25798 handle_command_reply: tid=0 2024-09-19T09:35:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.164+0000 7f148c0e2740 1 client.25798 shutdown 2024-09-19T09:35:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 msgr2=0x5558dcc35e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1474031c70 tx=0x7f1474031510 comp rx=0 tx=0).stop 2024-09-19T09:35:03.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 shutdown_connections 2024-09-19T09:35:03.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5558dcc368f0 0x5558dccab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5558dccad480 0x5558dccaf8a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5558dcc31e90 0x5558dcc363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 --2- 172.21.15.28:0/2655310018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558dcc1ece0 0x5558dcc35e70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 >> 172.21.15.28:0/2655310018 conn(0x5558dcc1aa30 msgr2=0x5558dcc2e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.167+0000 7f1462ffd640 0 client.25798 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.168+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 shutdown_connections 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.168+0000 7f148c0e2740 1 -- 172.21.15.28:0/2655310018 wait complete. 2024-09-19T09:35:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.168+0000 7f148c0e2740 20 client.25798 trim_cache size 0 max 16384 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "scrub active (5010 inodes in the stack)", 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "d6c3b6be-3fe0-4496-a752-e617b79d6306": { 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "d6c3b6be-3fe0-4496-a752-e617b79d6306", 2024-09-19T09:35:03.171 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive,force" 2024-09-19T09:35:03.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:35:03.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:35:03.172 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:35:03.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 msgr2=0x7f14700a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 0x7f14700a5890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f147c013e30 tx=0x7f147c019000 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 msgr2=0x7f148419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f146c00df40 tx=0x7f146c008740 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 shutdown_connections 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f14700a33d0 0x7f14700a5890 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f148410ef30 0x7f14841a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148410e580 0x7f148419f8a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 --2- 172.21.15.28:0/1774557479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f148410d9b0 0x7f148419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.171+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 >> 172.21.15.28:0/1774557479 conn(0x7f148406e140 msgr2=0x7f148410f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.172+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 shutdown_connections 2024-09-19T09:35:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:03.172+0000 7f1489d09640 1 -- 172.21.15.28:0/1774557479 wait complete. 2024-09-19T09:35:03.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:03 smithi028 ceph-mon[30972]: pgmap v409: 97 pgs: 97 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-09-19T09:35:03.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:03 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:03.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:03 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:03 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:03.504 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:03 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:03 smithi078 ceph-mon[35619]: pgmap v409: 97 pgs: 97 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:03 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:03 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:03 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:03 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:03 smithi078 podman[69049]: 2024-09-19 09:35:03.380777089 +0000 UTC m=+1.393251695 container create ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T09:35:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:03 smithi038 ceph-mon[35271]: pgmap v409: 97 pgs: 97 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-09-19T09:35:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:03 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:03 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:03 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:03 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:03.951 INFO:tasks.cephfs.filesystem:scrub status for tag:d6c3b6be-3fe0-4496-a752-e617b79d6306 - {'path': '/', 'tag': 'd6c3b6be-3fe0-4496-a752-e617b79d6306', 'options': 'recursive,force'} 2024-09-19T09:35:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.784+0000 7f3901ffb640 1 -- 172.21.15.28:0/1858970590 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac8ebcdd70 con 0x55ac8ece3db0 2024-09-19T09:35:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.784+0000 7f3901ffb640 10 client.16227 handle_command_reply: tid=0 2024-09-19T09:35:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.784+0000 7f3923224740 1 client.16227 shutdown 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.784+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 msgr2=0x55ac8ec72ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.784+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3910012870 tx=0x7f39100096a0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 shutdown_connections 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ac8ece3db0 0x55ac8ece61d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac8ec68370 0x55ac8ec72ac0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac8ec632e0 0x55ac8ec71ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 --2- 172.21.15.28:0/1858970590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac8ec629a0 0x55ac8ec71570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3901ffb640 0 client.16227 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 >> 172.21.15.28:0/1858970590 conn(0x55ac8ec5e010 msgr2=0x55ac8ec5d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 shutdown_connections 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 1 -- 172.21.15.28:0/1858970590 wait complete. 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.785+0000 7f3923224740 20 client.16227 trim_cache size 0 max 16384 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:35:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:35:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:35:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 msgr2=0x7f39040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 0x7f39040a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3910005fd0 tx=0x7f3910004ab0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 msgr2=0x7f39181983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f390800e9c0 tx=0x7f390800ee80 comp rx=0 tx=0).stop 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 shutdown_connections 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39040a3240 0x7f39040a5700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3918103720 0x7f391819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3918102d60 0x7f39181983f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 --2- 172.21.15.28:0/4078704816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3918101b70 0x7f3918197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 >> 172.21.15.28:0/4078704816 conn(0x7f39180fd2f0 msgr2=0x7f39181061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 shutdown_connections 2024-09-19T09:35:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:04.786+0000 7f392048b640 1 -- 172.21.15.28:0/4078704816 wait complete. 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:05 smithi078 ceph-mon[35619]: pgmap v410: 97 pgs: 97 active+clean; 60 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:05 smithi078 ceph-mon[35619]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:05 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 podman[69049]: 2024-09-19 09:35:05.434764225 +0000 UTC m=+3.447238823 container init ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 podman[69049]: 2024-09-19 09:35:05.437788083 +0000 UTC m=+3.450262683 container start ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T09:35:05.470 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 ceph-mds[69067]: -- 172.21.15.78:0/305518540 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55627b0f3860 con 0x55627b0d1400 2024-09-19T09:35:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:05 smithi028 ceph-mon[30972]: pgmap v410: 97 pgs: 97 active+clean; 60 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-09-19T09:35:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:05 smithi028 ceph-mon[30972]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:35:05.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:05 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:35:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:05 smithi038 ceph-mon[35271]: pgmap v410: 97 pgs: 97 active+clean; 60 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-09-19T09:35:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:05 smithi038 ceph-mon[35271]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:35:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:05 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:35:05.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[69063]: starting mds.l at 2024-09-19T09:35:05.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 bash[69049]: ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 2024-09-19T09:35:05.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:35:05 smithi078 systemd[1]: Started Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:35:07.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:07 smithi028 ceph-mon[30972]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 430 op/s 2024-09-19T09:35:07.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:07 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6832/686387781,v1:172.21.15.78:6833/686387781] up:boot 2024-09-19T09:35:07.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:07 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:07.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:07 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:35:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:07 smithi078 ceph-mon[35619]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 430 op/s 2024-09-19T09:35:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:07 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6832/686387781,v1:172.21.15.78:6833/686387781] up:boot 2024-09-19T09:35:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:07 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:07 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:35:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:07 smithi038 ceph-mon[35271]: pgmap v411: 97 pgs: 97 active+clean; 55 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 430 op/s 2024-09-19T09:35:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:07 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6832/686387781,v1:172.21.15.78:6833/686387781] up:boot 2024-09-19T09:35:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:07 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:07 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:35:09.038 INFO:tasks.workunit.client.0.smithi028.stdout: 42 0 555 0 403 0 1184 2024-09-19T09:35:09.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:09 smithi028 ceph-mon[30972]: pgmap v412: 97 pgs: 97 active+clean; 72 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 4.4 MiB/s wr, 857 op/s 2024-09-19T09:35:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:09 smithi078 ceph-mon[35619]: pgmap v412: 97 pgs: 97 active+clean; 72 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 4.4 MiB/s wr, 857 op/s 2024-09-19T09:35:09.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:09 smithi038 ceph-mon[35271]: pgmap v412: 97 pgs: 97 active+clean; 72 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 4.4 MiB/s wr, 857 op/s 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.072+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e15 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml auth{0=1} v94 pv772 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.534129+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.534129+0000 b2005 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348342100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e15:[<0x10000001dcd/article-18.xml v94>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e15:[<0x10000001dcd/article-18.xml v94>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.073+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ded [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml auth{0=1} v40 pv766 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.402133+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.402133+0000 b21608 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348341080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ded:[<0x10000001dcd/article-27.xml v40>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ded:[<0x10000001dcd/article-27.xml v40>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.074+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e0d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml auth{0=1} v82 pv778 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.505130+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.505130+0000 b5649 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348343180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e0d:[<0x10000001dcd/article-20.xml v82>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e0d:[<0x10000001dcd/article-20.xml v82>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.074+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dd9 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml auth{0=1} v16 pv782 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.252137+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.252137+0000 b20667 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348378000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dd9:[<0x10000001dcd/article-31.xml v16>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dd9:[<0x10000001dcd/article-31.xml v16>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.094+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e39 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml auth{0=1} v148 pv792 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.670125+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.670125+0000 b29685 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348379b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e39:[<0x10000001dcd/article-9.xml v148>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e39:[<0x10000001dcd/article-9.xml v148>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.096+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dfc [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml auth{0=1} v58 pv798 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.445132+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.445132+0000 b29241 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834837ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dfc:[<0x10000001dcd/article-24.xml v58>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dfc:[<0x10000001dcd/article-24.xml v58>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.359 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.101+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e51 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml auth{0=1} v184 pv810 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.760123+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.760123+0000 b5496 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348385080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e51:[<0x10000001dcd/article-3.xml v184>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e51:[<0x10000001dcd/article-3.xml v184>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.103+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e41 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml auth{0=1} v160 pv818 ap=4 s=0 n(v0 rc2024-09-19T09:35:01.700125+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.700125+0000 b1563 1=1+0) (iauth excl->sync) (ifile excl w=1) (ixattr excl->sync) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsxcrwb/pAsxLsXsxFsxcrwb/Fc@2},l=24926(-1) | ptrwaiter=1 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x558348386680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e41:[<0x10000001dcd/article-7.xml v160>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e41:[<0x10000001dcd/article-7.xml v160>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.134+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e3d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml auth{0=1} v154 pv1078 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.684125+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.684125+0000 b31554 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834838c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e3d:[<0x10000001dcd/article-8.xml v154>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e3d:[<0x10000001dcd/article-8.xml v154>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.137+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dd4 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml auth{0=1} v10 pv1090 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.238137+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.238137+0000 b10257 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834838e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dd4:[<0x10000001dcd/article-32.xml v10>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dd4:[<0x10000001dcd/article-32.xml v10>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.140+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e09 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml auth{0=1} v76 pv1092 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.491130+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.491130+0000 b19222 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834838ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e09:[<0x10000001dcd/article-21.xml v76>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e09:[<0x10000001dcd/article-21.xml v76>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.144+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dde [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml auth{0=1} v22 pv1094 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.271136+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.271136+0000 b12907 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834838f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dde:[<0x10000001dcd/article-30.xml v22>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dde:[<0x10000001dcd/article-30.xml v22>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.162+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e29 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml auth{0=1} v124 pv1096 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.614127+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.614127+0000 b31773 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834838f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e29:[<0x10000001dcd/article-13.xml v124>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e29:[<0x10000001dcd/article-13.xml v124>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.166+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e45 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml auth{0=1} v166 pv1098 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.714124+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.714124+0000 b30384 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834839c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e45:[<0x10000001dcd/article-6.xml v166>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e45:[<0x10000001dcd/article-6.xml v166>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.170+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e55 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml auth{0=1} v190 pv1100 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.776122+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.776122+0000 b16212 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834839c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e55:[<0x10000001dcd/article-2.xml v190>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e55:[<0x10000001dcd/article-2.xml v190>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.171+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dcf [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml auth{0=1} v4 pv1108 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.224138+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.224138+0000 b15613 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x55834839db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dcf:[<0x10000001dcd/article-33.xml v4>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dcf:[<0x10000001dcd/article-33.xml v4>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.360 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.188+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001de8 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml auth{0=1} v34 pv1120 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.387133+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.387133+0000 b12717 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483a6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001de8:[<0x10000001dcd/article-28.xml v34>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001de8:[<0x10000001dcd/article-28.xml v34>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.193+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e49 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml auth{0=1} v172 pv1140 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.730124+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.730124+0000 b2173 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483a9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e49:[<0x10000001dcd/article-5.xml v172>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e49:[<0x10000001dcd/article-5.xml v172>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.197+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001de3 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml auth{0=1} v28 pv1144 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.290136+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.290136+0000 b6832 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483b4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001de3:[<0x10000001dcd/article-29.xml v28>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001de3:[<0x10000001dcd/article-29.xml v28>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.209+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e05 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml auth{0=1} v70 pv1150 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.475131+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.475131+0000 b15796 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483b5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e05:[<0x10000001dcd/article-22.xml v70>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e05:[<0x10000001dcd/article-22.xml v70>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.218+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e25 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml auth{0=1} v118 pv1160 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.596127+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.596127+0000 b19876 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483b7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e25:[<0x10000001dcd/article-14.xml v118>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e25:[<0x10000001dcd/article-14.xml v118>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.221+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e31 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml auth{0=1} v136 pv1176 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.643126+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.643126+0000 b6010 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483c2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e31:[<0x10000001dcd/article-11.xml v136>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e31:[<0x10000001dcd/article-11.xml v136>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.223+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e1d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml auth{0=1} v106 pv1178 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.564128+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.564128+0000 b21114 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483c2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e1d:[<0x10000001dcd/article-16.xml v106>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e1d:[<0x10000001dcd/article-16.xml v106>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.224+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df2 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml auth{0=1} v46 pv1180 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.416132+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.416132+0000 b12440 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483c2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001df2:[<0x10000001dcd/article-26.xml v46>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001df2:[<0x10000001dcd/article-26.xml v46>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.226+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e35 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml auth{0=1} v142 pv1188 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.656126+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.656126+0000 b19199 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483cc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e35:[<0x10000001dcd/article-10.xml v142>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e35:[<0x10000001dcd/article-10.xml v142>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.228+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e4d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml auth{0=1} v178 pv1194 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.746123+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.746123+0000 b31169 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483cd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e4d:[<0x10000001dcd/article-4.xml v178>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e4d:[<0x10000001dcd/article-4.xml v178>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.230+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df7 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml auth{0=1} v52 pv1196 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.430132+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.430132+0000 b18590 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483cdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001df7:[<0x10000001dcd/article-25.xml v52>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001df7:[<0x10000001dcd/article-25.xml v52>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.233+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e21 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml auth{0=1} v112 pv1210 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.579128+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.579128+0000 b15968 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483d4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e21:[<0x10000001dcd/article-15.xml v112>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e21:[<0x10000001dcd/article-15.xml v112>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.361 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.236+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e11 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml auth{0=1} v88 pv1216 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.519129+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.519129+0000 b6233 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483d5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e11:[<0x10000001dcd/article-19.xml v88>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e11:[<0x10000001dcd/article-19.xml v88>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.362 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.238+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e2d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml auth{0=1} v130 pv1224 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.629126+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.629126+0000 b32274 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483d6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e2d:[<0x10000001dcd/article-12.xml v130>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e2d:[<0x10000001dcd/article-12.xml v130>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.362 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.240+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e01 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml auth{0=1} v64 pv1242 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.461131+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.461131+0000 b23331 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483ec100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e01:[<0x10000001dcd/article-23.xml v64>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e01:[<0x10000001dcd/article-23.xml v64>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:10.362 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:35:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:35:10.242+0000 7f3e4aa71640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e19 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml auth{0=1} v100 pv1244 ap=3 s=0 n(v0 rc2024-09-19T09:35:01.549129+0000 1=1+0)/n(v0 1=1+0)->n(v0 rc2024-09-19T09:35:01.549129+0000 b7721 1=1+0) (ifile excl w=2) cr={24926=0-4194304@1} caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=2 lock=1 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 authpin=1 scrubqueue=0 0x5583483ec680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001e19:[<0x10000001dcd/article-17.xml v100>,<0x10000000399/blog-40 v49049>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001e19:[<0x10000001dcd/article-17.xml v100>,<0x10000000399/blog-40 v51323>,<0x10000000208/blogtest_in v451>,<0x10000000201/src v2124>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v95>,<0x100000001fb/client.0 v89>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v102>,<0x10000000001/sv_1 v116>,<0x10000000000/qa v57>,<0x1/volumes v105>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: pgmap v413: 97 pgs: 97 active+clean; 103 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e15 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ded (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dd9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.230 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dfc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e51 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e41 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e3d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.231 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dd4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e09 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dde (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e29 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.232 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e45 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dcf (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001de8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.233 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001de3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e05 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e25 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e31 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.515 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e1d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001df2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e4d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.516 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001df7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e11 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e2d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e01 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.518 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:11.518 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001e19 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.518 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:11 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: pgmap v413: 97 pgs: 97 active+clean; 103 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e15 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ded (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dd9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dfc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e51 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e41 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e3d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dd4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e09 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dde (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e29 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e45 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dcf (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001de8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001de3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e05 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e25 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e31 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e1d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001df2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e4d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001df7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e11 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e2d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e01 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001e19 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:11 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: pgmap v413: 97 pgs: 97 active+clean; 103 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e15 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ded (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dd9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dfc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e51 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e41 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e3d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dd4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e09 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dde (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e29 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e45 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dcf (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.679 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001de8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001de3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e05 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e25 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e31 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e1d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001df2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e4d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001df7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e11 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e2d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e01 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:11.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:11.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001e19 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:35:11.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:11 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:13 smithi028 ceph-mon[30972]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.8 MiB/s wr, 1.31k op/s 2024-09-19T09:35:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:13 smithi078 ceph-mon[35619]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.8 MiB/s wr, 1.31k op/s 2024-09-19T09:35:13.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:13 smithi038 ceph-mon[35271]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.8 MiB/s wr, 1.31k op/s 2024-09-19T09:35:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:15 smithi028 ceph-mon[30972]: pgmap v415: 97 pgs: 97 active+clean; 127 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 1.46k op/s 2024-09-19T09:35:15.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:15 smithi078 ceph-mon[35619]: pgmap v415: 97 pgs: 97 active+clean; 127 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 1.46k op/s 2024-09-19T09:35:15.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:15 smithi038 ceph-mon[35271]: pgmap v415: 97 pgs: 97 active+clean; 127 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 1.46k op/s 2024-09-19T09:35:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:16 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:35:16.230+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:16 smithi028 ceph-mon[30972]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:16 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:35:16.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:16 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:16.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:16 smithi078 ceph-mon[35619]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:16.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:16 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:35:16.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:16 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:16.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-mon[35271]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:16.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] up:active 2024-09-19T09:35:16.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:17.286 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.948+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c3c [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml auth{0=1} v1142 ap=1 s=13597 n(v0 rc2024-09-19T09:34:59.849176+0000 b13597 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b25beb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c3c:[<0x10000001c35/article-59.xml v16>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c3c:[<0x10000001c35/article-59.xml v1142>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.949+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d91 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg auth{0=1} v1232 ap=1 s=14629 n(v0 rc2024-09-19T09:35:00.989144+0000 b14629 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2930000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d91:[<0x10000001c35/picture-13.jpg v454>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d91:[<0x10000001c35/picture-13.jpg v1232>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.949+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d7a [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg auth{0=1} v1234 ap=1 s=35284 n(v0 rc2024-09-19T09:35:00.940145+0000 b35284 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2930580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d7a:[<0x10000001c35/picture-18.jpg v424>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d7a:[<0x10000001c35/picture-18.jpg v1234>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.950+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cea [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml auth{0=1} v1236 ap=1 s=10607 n(v0 rc2024-09-19T09:35:00.521157+0000 b10607 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2930b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cea:[<0x10000001c35/article-18.xml v262>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cea:[<0x10000001c35/article-18.xml v1236>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.951+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d74 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg auth{0=1} v1240 ap=1 s=53062 n(v0 rc2024-09-19T09:35:00.927146+0000 b53062 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2931600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d74:[<0x10000001c35/picture-19.jpg v418>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d74:[<0x10000001c35/picture-19.jpg v1240>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.952+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c55 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml auth{0=1} v1242 ap=1 s=26161 n(v0 rc2024-09-19T09:34:59.938173+0000 b26161 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2931b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c55:[<0x10000001c35/article-51.xml v64>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c55:[<0x10000001c35/article-51.xml v1242>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.952+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cbd [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml auth{0=1} v1246 ap=1 s=10065 n(v0 rc2024-09-19T09:35:00.306163+0000 b10065 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2932680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cbd:[<0x10000001c35/article-27.xml v208>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cbd:[<0x10000001c35/article-27.xml v1246>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.953+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001db8 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg auth{0=1} v1248 ap=1 s=20110 n(v0 rc2024-09-19T09:35:01.151140+0000 b20110 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2932c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001db8:[<0x10000001c35/picture-4.jpg v508>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001db8:[<0x10000001c35/picture-4.jpg v1248>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.954+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dc3 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg auth{0=1} v1250 ap=1 s=39600 n(v0 rc2024-09-19T09:35:01.187139+0000 b39600 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2933180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dc3:[<0x10000001c35/picture-2.jpg v520>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dc3:[<0x10000001c35/picture-2.jpg v1250>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.287 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.956+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ce0 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml auth{0=1} v1252 ap=1 s=24330 n(v0 rc2024-09-19T09:35:00.462159+0000 b24330 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2933700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ce0:[<0x10000001c35/article-20.xml v250>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ce0:[<0x10000001c35/article-20.xml v1252>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.956+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d8d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg auth{0=1} v1210 ap=1 s=17049 n(v0 rc2024-09-19T09:35:00.980144+0000 b17049 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2878000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d8d:[<0x10000001c35/picture-14.jpg v448>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d8d:[<0x10000001c35/picture-14.jpg v1210>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.957+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c37 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml auth{0=1} v1212 ap=1 s=8797 n(v0 rc2024-09-19T09:34:59.826176+0000 b8797 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2878580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c37:[<0x10000001c35/article-61.xml v4>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c37:[<0x10000001c35/article-61.xml v1212>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.959+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c7a [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml auth{0=1} v1214 ap=1 s=4498 n(v0 rc2024-09-19T09:35:00.067170+0000 b4498 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2878b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c7a:[<0x10000001c35/article-41.xml v124>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c7a:[<0x10000001c35/article-41.xml v1214>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.959+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c6a [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml auth{0=1} v1216 ap=1 s=23199 n(v0 rc2024-09-19T09:35:00.011171+0000 b23199 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2879080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c6a:[<0x10000001c35/article-45.xml v100>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c6a:[<0x10000001c35/article-45.xml v1216>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.960+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001da9 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg auth{0=1} v1218 ap=1 s=42990 n(v0 rc2024-09-19T09:35:01.051142+0000 b42990 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2879600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001da9:[<0x10000001c35/picture-7.jpg v490>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001da9:[<0x10000001c35/picture-7.jpg v1218>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.288 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.961+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ca9 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml auth{0=1} v1220 ap=1 s=9600 n(v0 rc2024-09-19T09:35:00.235165+0000 b9600 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2879b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ca9:[<0x10000001c35/article-31.xml v184>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ca9:[<0x10000001c35/article-31.xml v1220>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.962+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001da1 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg auth{0=1} v1222 ap=1 s=64180 n(v0 rc2024-09-19T09:35:01.030143+0000 b64180 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b287a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001da1:[<0x10000001c35/picture-9.jpg v478>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001da1:[<0x10000001c35/picture-9.jpg v1222>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.963+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d9d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg auth{0=1} v1230 ap=1 s=11978 n(v0 rc2024-09-19T09:35:01.020143+0000 b11978 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b287b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d9d:[<0x10000001c35/picture-10.jpg v472>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d9d:[<0x10000001c35/picture-10.jpg v1230>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.964+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dc7 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg auth{0=1} v1188 ap=1 s=45266 n(v0 rc2024-09-19T09:35:01.204138+0000 b45266 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2836000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dc7:[<0x10000001c35/picture-1.jpg v526>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dc7:[<0x10000001c35/picture-1.jpg v1188>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.965+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d17 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml auth{0=1} v1192 ap=1 s=23900 n(v0 rc2024-09-19T09:35:00.714152+0000 b23900 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2836b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d17:[<0x10000001c35/article-9.xml v316>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d17:[<0x10000001c35/article-9.xml v1192>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.966+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c6e [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml auth{0=1} v1194 ap=1 s=7813 n(v0 rc2024-09-19T09:35:00.025171+0000 b7813 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2837080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c6e:[<0x10000001c35/article-44.xml v106>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c6e:[<0x10000001c35/article-44.xml v1194>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.967+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dae [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg auth{0=1} v1196 ap=1 s=57975 n(v0 rc2024-09-19T09:35:01.107141+0000 b57975 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2837600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dae:[<0x10000001c35/picture-6.jpg v496>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dae:[<0x10000001c35/picture-6.jpg v1196>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.293 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.968+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ccc [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml auth{0=1} v1198 ap=1 s=32221 n(v0 rc2024-09-19T09:35:00.363161+0000 b32221 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2837b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ccc:[<0x10000001c35/article-24.xml v226>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ccc:[<0x10000001c35/article-24.xml v1198>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.969+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d80 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg auth{0=1} v1206 ap=1 s=40517 n(v0 rc2024-09-19T09:35:00.954145+0000 b40517 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2839180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d80:[<0x10000001c35/picture-17.jpg v430>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d80:[<0x10000001c35/picture-17.jpg v1206>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.969+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d35 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml auth{0=1} v1208 ap=1 s=19529 n(v0 rc2024-09-19T09:35:00.790150+0000 b19529 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2839700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d35:[<0x10000001c35/article-3.xml v352>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d35:[<0x10000001c35/article-3.xml v1208>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.971+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d56 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg auth{0=1} v1166 ap=1 s=52150 n(v0 rc2024-09-19T09:35:00.862148+0000 b52150 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27aa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d56:[<0x10000001c35/picture-24.jpg v388>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d56:[<0x10000001c35/picture-24.jpg v1166>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.972+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d21 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml auth{0=1} v1168 ap=1 s=22320 n(v0 rc2024-09-19T09:35:00.742151+0000 b22320 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d21:[<0x10000001c35/article-7.xml v328>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d21:[<0x10000001c35/article-7.xml v1168>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.972+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c49 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml auth{0=1} v1174 ap=1 s=13916 n(v0 rc2024-09-19T09:34:59.896174+0000 b13916 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27ab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c49:[<0x10000001c35/article-55.xml v40>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c49:[<0x10000001c35/article-55.xml v1174>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.974+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d50 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg auth{0=1} v1176 ap=1 s=34617 n(v0 rc2024-09-19T09:35:00.849148+0000 b34617 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27abb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d50:[<0x10000001c35/picture-25.jpg v382>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d50:[<0x10000001c35/picture-25.jpg v1176>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.975+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d1c [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml auth{0=1} v1178 ap=1 s=23136 n(v0 rc2024-09-19T09:35:00.730151+0000 b23136 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27ac100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d1c:[<0x10000001c35/article-8.xml v322>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d1c:[<0x10000001c35/article-8.xml v1178>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.976+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001da5 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg auth{0=1} v1184 ap=1 s=15905 n(v0 rc2024-09-19T09:35:01.041143+0000 b15905 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001da5:[<0x10000001c35/picture-8.jpg v484>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001da5:[<0x10000001c35/picture-8.jpg v1184>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.977+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d49 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg auth{0=1} v1186 ap=1 s=17163 n(v0 rc2024-09-19T09:35:00.836148+0000 b17163 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b27ad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d49:[<0x10000001c35/picture-26.jpg v376>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d49:[<0x10000001c35/picture-26.jpg v1186>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.294 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.977+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ca4 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml auth{0=1} v1120 ap=1 s=25769 n(v0 rc2024-09-19T09:35:00.218165+0000 b25769 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2202000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ca4:[<0x10000001c35/article-32.xml v178>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ca4:[<0x10000001c35/article-32.xml v1120>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.979+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cdb [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml auth{0=1} v1122 ap=1 s=24324 n(v0 rc2024-09-19T09:35:00.431160+0000 b24324 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2202580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cdb:[<0x10000001c35/article-21.xml v244>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cdb:[<0x10000001c35/article-21.xml v1122>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.980+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cae [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml auth{0=1} v1124 ap=1 s=15780 n(v0 rc2024-09-19T09:35:00.252164+0000 b15780 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2202b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cae:[<0x10000001c35/article-30.xml v190>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cae:[<0x10000001c35/article-30.xml v1124>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.980+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d03 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml auth{0=1} v1128 ap=1 s=15236 n(v0 rc2024-09-19T09:35:00.625154+0000 b15236 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2203600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d03:[<0x10000001c35/article-13.xml v292>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d03:[<0x10000001c35/article-13.xml v1128>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.984+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c58 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml auth{0=1} v1132 ap=1 s=8840 n(v0 rc2024-09-19T09:34:59.948173+0000 b8840 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2204100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c58:[<0x10000001c35/article-50.xml v70>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c58:[<0x10000001c35/article-50.xml v1132>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.985+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d26 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml auth{0=1} v1134 ap=1 s=28142 n(v0 rc2024-09-19T09:35:00.755151+0000 b28142 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2204680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d26:[<0x10000001c35/article-6.xml v334>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d26:[<0x10000001c35/article-6.xml v1134>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.986+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d3a [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml auth{0=1} v1136 ap=1 s=29963 n(v0 rc2024-09-19T09:35:00.802149+0000 b29963 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2204c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d3a:[<0x10000001c35/article-2.xml v358>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d3a:[<0x10000001c35/article-2.xml v1136>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.987+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c46 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml auth{0=1} v1144 ap=1 s=26342 n(v0 rc2024-09-19T09:34:59.882175+0000 b26342 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b278a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c46:[<0x10000001c35/article-56.xml v34>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c46:[<0x10000001c35/article-56.xml v1144>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.295 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.987+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c72 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml auth{0=1} v1152 ap=1 s=24807 n(v0 rc2024-09-19T09:35:00.039170+0000 b24807 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b278b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c72:[<0x10000001c35/article-43.xml v112>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c72:[<0x10000001c35/article-43.xml v1152>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.989+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c9f [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml auth{0=1} v1154 ap=1 s=15371 n(v0 rc2024-09-19T09:35:00.200166+0000 b15371 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b278bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c9f:[<0x10000001c35/article-33.xml v172>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c9f:[<0x10000001c35/article-33.xml v1154>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.990+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001db3 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg auth{0=1} v1156 ap=1 s=33144 n(v0 rc2024-09-19T09:35:01.131140+0000 b33144 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b278c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001db3:[<0x10000001c35/picture-5.jpg v502>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001db3:[<0x10000001c35/picture-5.jpg v1156>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.991+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c42 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml auth{0=1} v1160 ap=1 s=15439 n(v0 rc2024-09-19T09:34:59.870175+0000 b15439 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b278cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c42:[<0x10000001c35/article-57.xml v28>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c42:[<0x10000001c35/article-57.xml v1160>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.993+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c40 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml auth{0=1} v1098 ap=1 s=23509 n(v0 rc2024-09-19T09:34:59.858175+0000 b23509 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b21ec000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c40:[<0x10000001c35/article-58.xml v22>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c40:[<0x10000001c35/article-58.xml v1098>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.993+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c4f [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml auth{0=1} v1100 ap=1 s=16188 n(v0 rc2024-09-19T09:34:59.917174+0000 b16188 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b21ec580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c4f:[<0x10000001c35/article-53.xml v52>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c4f:[<0x10000001c35/article-53.xml v1100>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.994+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cb8 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml auth{0=1} v1106 ap=1 s=17713 n(v0 rc2024-09-19T09:35:00.288163+0000 b17713 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b21ed600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cb8:[<0x10000001c35/article-28.xml v202>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cb8:[<0x10000001c35/article-28.xml v1106>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.995+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c52 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml auth{0=1} v1110 ap=1 s=4582 n(v0 rc2024-09-19T09:34:59.928173+0000 b4582 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b21ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c52:[<0x10000001c35/article-52.xml v58>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c52:[<0x10000001c35/article-52.xml v1110>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.996+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dbe [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg auth{0=1} v1116 ap=1 s=15628 n(v0 rc2024-09-19T09:35:01.169139+0000 b15628 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b21ef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001dbe:[<0x10000001c35/picture-3.jpg v514>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001dbe:[<0x10000001c35/picture-3.jpg v1116>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.998+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d99 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg auth{0=1} v1254 ap=1 s=41946 n(v0 rc2024-09-19T09:35:01.010144+0000 b41946 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d99:[<0x10000001c35/picture-11.jpg v466>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d99:[<0x10000001c35/picture-11.jpg v1254>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.296 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.998+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d2b [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml auth{0=1} v1256 ap=1 s=17866 n(v0 rc2024-09-19T09:35:00.767150+0000 b17866 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d2b:[<0x10000001c35/article-5.xml v340>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d2b:[<0x10000001c35/article-5.xml v1256>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:16.999+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c9a [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml auth{0=1} v1258 ap=1 s=27865 n(v0 rc2024-09-19T09:35:00.182166+0000 b27865 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c9a:[<0x10000001c35/article-34.xml v166>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c9a:[<0x10000001c35/article-34.xml v1258>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.001+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cb3 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml auth{0=1} v1264 ap=1 s=30440 n(v0 rc2024-09-19T09:35:00.271164+0000 b30440 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cb3:[<0x10000001c35/article-29.xml v196>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cb3:[<0x10000001c35/article-29.xml v1264>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.001+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c39 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml auth{0=1} v1266 ap=1 s=15137 n(v0 rc2024-09-19T09:34:59.838176+0000 b15137 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c39:[<0x10000001c35/article-60.xml v10>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c39:[<0x10000001c35/article-60.xml v1266>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.002+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cd6 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml auth{0=1} v1268 ap=1 s=1105 n(v0 rc2024-09-19T09:35:00.406160+0000 b1105 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cd6:[<0x10000001c35/article-22.xml v238>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cd6:[<0x10000001c35/article-22.xml v1268>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.003+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c5b [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml auth{0=1} v1272 ap=1 s=30538 n(v0 rc2024-09-19T09:34:59.955173+0000 b30538 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b299d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c5b:[<0x10000001c35/article-49.xml v76>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c5b:[<0x10000001c35/article-49.xml v1272>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.004+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d5c [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg auth{0=1} v1276 ap=1 s=57256 n(v0 rc2024-09-19T09:35:00.875147+0000 b57256 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c74000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d5c:[<0x10000001c35/picture-23.jpg v394>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d5c:[<0x10000001c35/picture-23.jpg v1276>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.006+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cfe [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml auth{0=1} v1278 ap=1 s=1287 n(v0 rc2024-09-19T09:35:00.605155+0000 b1287 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c74580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cfe:[<0x10000001c35/article-14.xml v286>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cfe:[<0x10000001c35/article-14.xml v1278>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.007+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c66 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml auth{0=1} v1280 ap=1 s=10293 n(v0 rc2024-09-19T09:34:59.996171+0000 b10293 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c74b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c66:[<0x10000001c35/article-46.xml v94>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c66:[<0x10000001c35/article-46.xml v1280>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.007+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c62 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml auth{0=1} v1282 ap=1 s=17603 n(v0 rc2024-09-19T09:34:59.982172+0000 b17603 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c75080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c62:[<0x10000001c35/article-47.xml v88>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c62:[<0x10000001c35/article-47.xml v1282>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.009+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d62 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg auth{0=1} v1284 ap=1 s=61397 n(v0 rc2024-09-19T09:35:00.888147+0000 b61397 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c75600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d62:[<0x10000001c35/picture-22.jpg v400>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d62:[<0x10000001c35/picture-22.jpg v1284>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.010+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d0d [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml auth{0=1} v1286 ap=1 s=7044 n(v0 rc2024-09-19T09:35:00.673153+0000 b7044 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c75b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d0d:[<0x10000001c35/article-11.xml v304>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d0d:[<0x10000001c35/article-11.xml v1286>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.297 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.011+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c4c [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml auth{0=1} v1290 ap=1 s=11691 n(v0 rc2024-09-19T09:34:59.906174+0000 b11691 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c76680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c4c:[<0x10000001c35/article-54.xml v46>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c4c:[<0x10000001c35/article-54.xml v1290>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.011+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf4 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml auth{0=1} v1288 ap=1 s=23992 n(v0 rc2024-09-19T09:35:00.566156+0000 b23992 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c76100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cf4:[<0x10000001c35/article-16.xml v274>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cf4:[<0x10000001c35/article-16.xml v1288>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.012+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cc2 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml auth{0=1} v1292 ap=1 s=15584 n(v0 rc2024-09-19T09:35:00.323162+0000 b15584 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c76c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cc2:[<0x10000001c35/article-26.xml v214>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cc2:[<0x10000001c35/article-26.xml v1292>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.012+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d89 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg auth{0=1} v1294 ap=1 s=36952 n(v0 rc2024-09-19T09:35:00.972145+0000 b36952 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c77180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d89:[<0x10000001c35/picture-15.jpg v442>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d89:[<0x10000001c35/picture-15.jpg v1294>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.012+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d12 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml auth{0=1} v1296 ap=1 s=27099 n(v0 rc2024-09-19T09:35:00.697152+0000 b27099 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c77700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d12:[<0x10000001c35/article-10.xml v310>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d12:[<0x10000001c35/article-10.xml v1296>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.012+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c5e [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml auth{0=1} v1300 ap=1 s=6227 n(v0 rc2024-09-19T09:34:59.968172+0000 b6227 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c80580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c5e:[<0x10000001c35/article-48.xml v82>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c5e:[<0x10000001c35/article-48.xml v1300>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.013+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d95 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg auth{0=1} v1298 ap=1 s=49595 n(v0 rc2024-09-19T09:35:01.000144+0000 b49595 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c80000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d95:[<0x10000001c35/picture-12.jpg v460>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d95:[<0x10000001c35/picture-12.jpg v1298>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.013+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d30 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml auth{0=1} v1304 ap=1 s=11999 n(v0 rc2024-09-19T09:35:00.778150+0000 b11999 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c81080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d30:[<0x10000001c35/article-4.xml v346>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d30:[<0x10000001c35/article-4.xml v1304>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.013+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cc7 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml auth{0=1} v1306 ap=1 s=18673 n(v0 rc2024-09-19T09:35:00.343162+0000 b18673 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c81600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cc7:[<0x10000001c35/article-25.xml v220>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cc7:[<0x10000001c35/article-25.xml v1306>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.014+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c86 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml auth{0=1} v1308 ap=1 s=18440 n(v0 rc2024-09-19T09:35:00.114168+0000 b18440 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c81b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c86:[<0x10000001c35/article-38.xml v142>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c86:[<0x10000001c35/article-38.xml v1308>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.014+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c95 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml auth{0=1} v1314 ap=1 s=16776 n(v0 rc2024-09-19T09:35:00.164167+0000 b16776 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c82c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c95:[<0x10000001c35/article-35.xml v160>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c95:[<0x10000001c35/article-35.xml v1314>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.298 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.014+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d68 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg auth{0=1} v1312 ap=1 s=38629 n(v0 rc2024-09-19T09:35:00.901147+0000 b38629 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d68:[<0x10000001c35/picture-21.jpg v406>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d68:[<0x10000001c35/picture-21.jpg v1312>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.015+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c7e [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml auth{0=1} v1318 ap=1 s=30290 n(v0 rc2024-09-19T09:35:00.081169+0000 b30290 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c83700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c7e:[<0x10000001c35/article-40.xml v130>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c7e:[<0x10000001c35/article-40.xml v1318>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.015+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf9 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml auth{0=1} v1320 ap=1 s=32313 n(v0 rc2024-09-19T09:35:00.586155+0000 b32313 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c8c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cf9:[<0x10000001c35/article-15.xml v280>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cf9:[<0x10000001c35/article-15.xml v1320>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.015+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ce5 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml auth{0=1} v1328 ap=1 s=12045 n(v0 rc2024-09-19T09:35:00.493158+0000 b12045 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c8d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ce5:[<0x10000001c35/article-19.xml v256>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001ce5:[<0x10000001c35/article-19.xml v1328>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.015+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d6e [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg auth{0=1} v1330 ap=1 s=14606 n(v0 rc2024-09-19T09:35:00.914146+0000 b14606 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c8db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d6e:[<0x10000001c35/picture-20.jpg v412>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d6e:[<0x10000001c35/picture-20.jpg v1330>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.015+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c76 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml auth{0=1} v1334 ap=1 s=14901 n(v0 rc2024-09-19T09:35:00.054170+0000 b14901 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c8e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c76:[<0x10000001c35/article-42.xml v118>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c76:[<0x10000001c35/article-42.xml v1334>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c8b [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml auth{0=1} v1342 ap=1 s=16353 n(v0 rc2024-09-19T09:35:00.131168+0000 b16353 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c8b:[<0x10000001c35/article-37.xml v148>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c8b:[<0x10000001c35/article-37.xml v1342>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d44 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg auth{0=1} v1344 ap=1 s=54433 n(v0 rc2024-09-19T09:35:00.824149+0000 b54433 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d44:[<0x10000001c35/picture-27.jpg v370>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d44:[<0x10000001c35/picture-27.jpg v1344>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c82 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml auth{0=1} v1346 ap=1 s=21450 n(v0 rc2024-09-19T09:35:00.096169+0000 b21450 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c82:[<0x10000001c35/article-39.xml v136>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c82:[<0x10000001c35/article-39.xml v1346>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001c90 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml auth{0=1} v1348 ap=1 s=32750 n(v0 rc2024-09-19T09:35:00.147167+0000 b32750 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001c90:[<0x10000001c35/article-36.xml v154>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001c90:[<0x10000001c35/article-36.xml v1348>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d08 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml auth{0=1} v1350 ap=1 s=25786 n(v0 rc2024-09-19T09:35:00.649153+0000 b25786 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d08:[<0x10000001c35/article-12.xml v298>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d08:[<0x10000001c35/article-12.xml v1350>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.016+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d3f [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml auth{0=1} v1352 ap=1 s=14189 n(v0 rc2024-09-19T09:35:00.813149+0000 b14189 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d3f:[<0x10000001c35/article-1.xml v364>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d3f:[<0x10000001c35/article-1.xml v1352>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.299 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.017+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d85 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg auth{0=1} v1360 ap=1 s=40367 n(v0 rc2024-09-19T09:35:00.965145+0000 b40367 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2c9d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d85:[<0x10000001c35/picture-16.jpg v436>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001d85:[<0x10000001c35/picture-16.jpg v1360>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.300 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.017+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cd1 [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml auth{0=1} v1368 ap=1 s=19727 n(v0 rc2024-09-19T09:35:00.382161+0000 b19727 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2cacb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cd1:[<0x10000001c35/article-23.xml v232>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cd1:[<0x10000001c35/article-23.xml v1368>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.300 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:35:17.017+0000 7ff435a8e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cef [2,head] /volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml auth{0=1} v1370 ap=1 s=16420 n(v0 rc2024-09-19T09:35:00.545156+0000 b16420 1=1+0) caps={24926=pAsLsXsFsc/-@2},l=24926(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5651b2cad080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001cef:[<0x10000001c35/article-17.xml v268>,<0x10000000399/blog-38 v47327>,<0x10000000208/blogtest_in v432>,<0x10000000201/src v2119>,<0x10000000006/blogbench-1.0 v1845>,<0x10000000005/tmp v93>,<0x100000001fb/client.0 v86>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v99>,<0x10000000001/sv_1 v114>,<0x10000000000/qa v107>,<0x1/volumes v103>]//[]","memoryvalue":"(3)0x10000001cef:[<0x10000001c35/article-17.xml v1370>,<0x10000000399/blog-38 v51341>,<0x10000000208/blogtest_in v500>,<0x10000000201/src v2142>,<0x10000000006/blogbench-1.0 v1870>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v98>,<0x100000001fa/ed69f18f-117b-456d-add4-9360702cc391 v111>,<0x10000000001/sv_1 v124>,<0x10000000000/qa v57>,<0x1/volumes v113>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-19T09:35:17.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:17 smithi078 ceph-mon[35619]: pgmap v416: 97 pgs: 97 active+clean; 135 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 1.45k op/s 2024-09-19T09:35:17.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:17 smithi038 ceph-mon[35271]: pgmap v416: 97 pgs: 97 active+clean; 135 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 1.45k op/s 2024-09-19T09:35:17.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:17 smithi028 ceph-mon[30972]: pgmap v416: 97 pgs: 97 active+clean; 135 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 1.45k op/s 2024-09-19T09:35:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 -- 172.21.15.28:0/1899643460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb40761d0 msgr2=0x7f0cb4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1899643460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb40761d0 0x7f0cb4074620 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009900 tx=0x7f0cb002a480 comp rx=0 tx=0).stop 2024-09-19T09:35:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 -- 172.21.15.28:0/1899643460 shutdown_connections 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1899643460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb4074bf0 0x7f0cb4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1899643460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb41099f0 0x7f0cb410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1899643460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb40761d0 0x7f0cb4074620 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.883+0000 7f0cbb978640 1 -- 172.21.15.28:0/1899643460 >> 172.21.15.28:0/1899643460 conn(0x7f0cb40fd2a0 msgr2=0x7f0cb40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- 172.21.15.28:0/1899643460 shutdown_connections 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- 172.21.15.28:0/1899643460 wait complete. 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 Processor -- start 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- start start 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 0x7f0cb419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb40761d0 0x7f0cb419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 0x7f0cb41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410f9f0 con 0x7f0cb41099f0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410f870 con 0x7f0cb40761d0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cbb978640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410fb70 con 0x7f0cb4074bf0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 0x7f0cb419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.884+0000 7f0cb9f15640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 0x7f0cb41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 0x7f0cb419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:48898/0 (socket says 172.21.15.28:48898) 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9714640 1 -- 172.21.15.28:0/1122970574 learned_addr learned my addr 172.21.15.28:0/1122970574 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb40761d0 0x7f0cb419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 -- 172.21.15.28:0/1122970574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 msgr2=0x7f0cb419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 0x7f0cb419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 -- 172.21.15.28:0/1122970574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb40761d0 msgr2=0x7f0cb419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb40761d0 0x7f0cb419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 -- 172.21.15.28:0/1122970574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a1cf0 con 0x7f0cb41099f0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 0x7f0cb41a1680 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009fd0 tx=0x7f0cac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0caa7fc640 1 -- 172.21.15.28:0/1122970574 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00ce20 con 0x7f0cb41099f0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0caa7fc640 1 -- 172.21.15.28:0/1122970574 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cac005020 con 0x7f0cb41099f0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a1fe0 con 0x7f0cb41099f0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0caa7fc640 1 -- 172.21.15.28:0/1122970574 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac010720 con 0x7f0cb41099f0 2024-09-19T09:35:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.885+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41a24a0 con 0x7f0cb41099f0 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.887+0000 7f0caa7fc640 1 -- 172.21.15.28:0/1122970574 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0cac005540 con 0x7f0cb41099f0 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0cbe7ee740 1 Processor -- start 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0cbe7ee740 1 -- start start 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0caa7fc640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 0x7f0ca00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.888+0000 7f0caa7fc640 1 -- 172.21.15.28:0/1122970574 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0cac0cff50 con 0x7f0cb41099f0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d65f0192f0 con 0x55d65f0b59a0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d65efc1270 con 0x55d65f0bb370 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d65f0152b0 con 0x55d65f0b62e0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 0x7f0ca00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45524/0 (socket says 172.21.15.28:45524) 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 -- 172.21.15.28:0/3636588429 learned_addr learned my addr 172.21.15.28:0/3636588429 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 -- 172.21.15.28:0/3636588429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 msgr2=0x55d65f0bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 -- 172.21.15.28:0/3636588429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 msgr2=0x55d65f0bd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0bd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 -- 172.21.15.28:0/3636588429 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65efc0340 con 0x55d65f0b59a0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009a20 tx=0x7f0cb0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed40d632c25d8798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 0x7f0ca00a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b620 tx=0x7f0ca400baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0c9f7fe640 1 -- 172.21.15.28:0/3636588429 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0040070 con 0x55d65f0b59a0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0c9f7fe640 1 -- 172.21.15.28:0/3636588429 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb0005030 con 0x55d65f0b59a0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.889+0000 7f0c9f7fe640 1 -- 172.21.15.28:0/3636588429 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0035040 con 0x55d65f0b59a0 2024-09-19T09:35:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3636588429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 msgr2=0x55d65f0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009a20 tx=0x7f0cb0005e80 comp rx=0 tx=0).stop 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3636588429 shutdown_connections 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3636588429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0b5da0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3636588429 >> 172.21.15.28:0/3636588429 conn(0x55d65f0b1010 msgr2=0x55d65f0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3636588429 shutdown_connections 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.890+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3636588429 wait complete. 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 -- start start 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 Processor -- start 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d65f0be2e0 con 0x55d65f0b59a0 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d65efa2290 con 0x55d65f0bb370 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d65f0be460 con 0x55d65f0b62e0 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cbe7ee740 10 client.0 _ll_register_callbacks cb 0x55d65efd3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45530/0 (socket says 172.21.15.28:45530) 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cb8f13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cb9714640 1 -- 172.21.15.28:0/3356202183 learned_addr learned my addr 172.21.15.28:0/3356202183 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.891+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 -- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 msgr2=0x55d65f0c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 -- 172.21.15.28:0/3356202183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 msgr2=0x55d65f0c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 -- 172.21.15.28:0/3356202183 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65ed2c210 con 0x55d65f0b59a0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cb9714640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005ab0 tx=0x7f0cb002ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff049bfb932ae291 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0040030 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0cb00318f0 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb0005030 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cb0035070 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cbe7ee740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d65edf8620 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.892+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb0005030 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f0cb0003710 con 0x55d65f0b59a0 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 20 client.16257 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f0cb460 0x55d65f0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 4 client.16257 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65f026a20 con 0x55d65f0cb460 2024-09-19T09:35:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f0cb460 0x55d65f0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0c9cff9640 10 client.16257 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.893+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f0cb460 0x55d65f0cd870 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e87ffec4b74378 server_cookie=d235bbf842bbe772 in_seq=0 out_seq=0 2024-09-19T09:35:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0c9cff9640 1 -- 172.21.15.28:0/3356202183 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65f026a20 con 0x55d65f0cb460 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0c9cff9640 10 client.16257 handle_command_reply: tid=0 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 client.16257 shutdown 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 msgr2=0x55d65f0bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005ab0 tx=0x7f0cb002ac30 comp rx=0 tx=0).stop 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 shutdown_connections 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0bb370 0x55d65f0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f0cb460 0x55d65f0cd870 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/3356202183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b59a0 0x55d65f0bfb90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0c9cff9640 0 client.16257 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.896+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 >> 172.21.15.28:0/3356202183 conn(0x55d65f0b1010 msgr2=0x55d65f0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.897+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 shutdown_connections 2024-09-19T09:35:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.897+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/3356202183 wait complete. 2024-09-19T09:35:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.897+0000 7f0cbe7ee740 20 client.16257 trim_cache size 0 max 16384 2024-09-19T09:35:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 Processor -- start 2024-09-19T09:35:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 -- start start 2024-09-19T09:35:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0bb370 0x55d65f0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41ab160 con 0x55d65f0b62e0 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41acd60 con 0x55d65f0b59a0 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41ab830 con 0x55d65f0bb370 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cb8f13640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cb8f13640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45540/0 (socket says 172.21.15.28:45540) 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.912+0000 7f0cb8f13640 1 -- 172.21.15.28:0/1082827724 learned_addr learned my addr 172.21.15.28:0/1082827724 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0bb370 0x55d65f0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb9714640 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 -- 172.21.15.28:0/1082827724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0bb370 msgr2=0x55d65f0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0bb370 0x55d65f0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 -- 172.21.15.28:0/1082827724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 msgr2=0x55d65f0c0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 -- 172.21.15.28:0/1082827724 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65f0cdef0 con 0x55d65f0b62e0 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0cb8f13640 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40058a0 tx=0x7f0ca4021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf283e22d97654b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0c9effd640 1 -- 172.21.15.28:0/1082827724 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4029420 con 0x55d65f0b62e0 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0c9effd640 1 -- 172.21.15.28:0/1082827724 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca402d030 con 0x55d65f0b62e0 2024-09-19T09:35:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.913+0000 7f0c9effd640 1 -- 172.21.15.28:0/1082827724 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca401fa30 con 0x55d65f0b62e0 2024-09-19T09:35:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/1082827724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 msgr2=0x55d65f0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40058a0 tx=0x7f0ca4021040 comp rx=0 tx=0).stop 2024-09-19T09:35:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/1082827724 shutdown_connections 2024-09-19T09:35:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0bb370 0x55d65f0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0b62e0 0x55d65f0cd850 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/1082827724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/1082827724 >> 172.21.15.28:0/1082827724 conn(0x55d65f0b1010 msgr2=0x55d65f0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/1082827724 shutdown_connections 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/1082827724 wait complete. 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 -- start start 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.914+0000 7f0cbe7ee740 1 Processor -- start 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41ac5d0 con 0x55d65f0bb370 2024-09-19T09:35:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41abf00 con 0x55d65f0b59a0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41a9140 con 0x55d65f0b62e0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 10 client.0 _ll_register_callbacks cb 0x7f0cac098120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9714640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45544/0 (socket says 172.21.15.28:45544) 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb8f13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 -- 172.21.15.28:0/2068189902 learned_addr learned my addr 172.21.15.28:0/2068189902 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 -- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 msgr2=0x55d65f0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 -- 172.21.15.28:0/2068189902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 msgr2=0x55d65f0c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 -- 172.21.15.28:0/2068189902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65f0c6020 con 0x55d65f0bb370 2024-09-19T09:35:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cb9f15640 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0989a0 tx=0x7f0cac09cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d8f6be3efb8d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac07c070 con 0x55d65f0bb370 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0cac0a4d20 con 0x55d65f0bb370 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca402d030 con 0x55d65f0bb370 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cac0b9020 con 0x55d65f0bb370 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 10 client.16263 fetch_fsmap learned FSMap version 32 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.915+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d65f0c7400 con 0x55d65f0bb370 2024-09-19T09:35:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca402d030 con 0x55d65f0bb370 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f0cac07f070 con 0x55d65f0bb370 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 10 client.16263 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 20 client.16263 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f136980 0x55d65f138da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 4 client.16263 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d65f020d70 con 0x55d65f136980 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cb9714640 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f136980 0x55d65f138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0c9cff9640 10 client.16263 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:17.916+0000 7f0cb9714640 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f136980 0x55d65f138da0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7252d262ed9b3721 server_cookie=dae71d898aef5746 in_seq=0 out_seq=0 2024-09-19T09:35:18.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c3c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d91 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cea (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d74 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cbd (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001db8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dc3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ce0 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.609 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d8d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c37 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c6a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001da9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ca9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001da1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d9d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.611 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d17 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.612 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.613 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.614 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ccc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d80 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.615 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d56 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.616 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d50 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d1c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001da5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ca4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.617 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cdb (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d03 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c58 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d26 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.618 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d3a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c46 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c72 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c9f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.619 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001db3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c42 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c40 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c4f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cb8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.620 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c52 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001dbe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d99 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d2b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c9a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.621 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cb3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cd6 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c5b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.622 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d5c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cfe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c66 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.623 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c4c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cf4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cc2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d89 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d12 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c5e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.624 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d30 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c86 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d68 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c7e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cf9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001ce5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.625 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c76 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c8b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d44 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c82 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001c90 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d08 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d3f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001d85 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cd1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: Scrub error on inode 0x10000001cef (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.626 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:18 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c3c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d91 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cea (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d74 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cbd (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001db8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dc3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ce0 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d8d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.678 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c37 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c6a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001da9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.680 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ca9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001da1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d9d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.681 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d17 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ccc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.682 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d80 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d56 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.683 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d50 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d1c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001da5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.684 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ca4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cdb (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d03 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.685 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c58 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d26 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d3a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c46 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.686 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c72 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c9f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001db3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.687 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c42 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c40 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.688 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c4f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cb8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.689 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c52 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001dbe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.690 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d99 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d2b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c9a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.691 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cb3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cd6 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c5b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.692 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d5c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cfe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c66 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.693 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c4c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cf4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cc2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d89 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d12 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c5e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.694 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d30 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c86 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d68 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c7e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cf9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001ce5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c76 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c8b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d44 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c82 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.695 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001c90 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d08 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d3f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001d85 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cd1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: Scrub error on inode 0x10000001cef (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.696 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:18 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c3c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d91 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cea (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d74 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c55 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cbd (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001db8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dc3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ce0 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.767 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d8d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.767 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.767 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c37 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c7a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.768 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c6a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001da9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ca9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001da1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d9d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.769 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d17 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.770 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ccc (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d80 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d35 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d56 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d21 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d50 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d1c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.772 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001da5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d49 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.773 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ca4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cdb (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cae (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d03 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.774 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c58 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d26 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d3a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c46 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.775 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c72 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c9f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001db3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c42 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c40 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.776 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c4f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cb8 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c52 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001dbe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d99 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d2b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c9a (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cb3 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c39 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.778 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cd6 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c5b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d5c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cfe (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.779 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c66 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d62 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d0d (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.780 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c4c (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cf4 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cc2 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d89 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d12 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c5e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d30 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cc7 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c86 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c95 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.781 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d68 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c7e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cf9 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001ce5 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d6e (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c76 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c8b (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d44 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c82 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001c90 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d08 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d3f (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001d85 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.782 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cd1 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.783 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: Scrub error on inode 0x10000001cef (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.h log and `damage ls` output for details 2024-09-19T09:35:18.783 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:18 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:19.038 INFO:tasks.workunit.client.0.smithi028.stdout: 52 0 486 0 487 0 1522 2024-09-19T09:35:19.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:19 smithi078 ceph-mon[35619]: pgmap v417: 97 pgs: 97 active+clean; 145 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-19T09:35:19.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:19 smithi038 ceph-mon[35271]: pgmap v417: 97 pgs: 97 active+clean; 145 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-19T09:35:19.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:19 smithi028 ceph-mon[30972]: pgmap v417: 97 pgs: 97 active+clean; 145 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-19T09:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0c9cff9640 1 -- 172.21.15.28:0/2068189902 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d65f020d70 con 0x55d65f136980 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0c9cff9640 10 client.16263 handle_command_reply: tid=0 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 client.16263 shutdown 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 msgr2=0x55d65f0c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0989a0 tx=0x7f0cac09cd80 comp rx=0 tx=0).stop 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 shutdown_connections 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55d65f136980 0x55d65f138da0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d65f0bb370 0x55d65f0c5a10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0c9cff9640 0 client.16263 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d65f0b62e0 0x55d65f0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 --2- 172.21.15.28:0/2068189902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d65f0b59a0 0x55d65f0c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 >> 172.21.15.28:0/2068189902 conn(0x55d65f0b1010 msgr2=0x55d65f0b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 shutdown_connections 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.412+0000 7f0cbe7ee740 1 -- 172.21.15.28:0/2068189902 wait complete. 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbe7ee740 20 client.16263 trim_cache size 0 max 16384 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:35:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:35:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 msgr2=0x7f0ca00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 0x7f0ca00a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b620 tx=0x7f0ca400baf0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 msgr2=0x7f0cb41a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 0x7f0cb41a1680 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009fd0 tx=0x7f0cac00ef10 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 shutdown_connections 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0ca00a3240 0x7f0ca00a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cb40761d0 0x7f0cb419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0cb41099f0 0x7f0cb41a1680 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 --2- 172.21.15.28:0/1122970574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cb4074bf0 0x7f0cb419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 >> 172.21.15.28:0/1122970574 conn(0x7f0cb40fd2a0 msgr2=0x7f0cb4105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 shutdown_connections 2024-09-19T09:35:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:21.413+0000 7f0cbb978640 1 -- 172.21.15.28:0/1122970574 wait complete. 2024-09-19T09:35:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:21 smithi078 ceph-mon[35619]: pgmap v418: 97 pgs: 97 active+clean; 149 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2024-09-19T09:35:21.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:21 smithi038 ceph-mon[35271]: pgmap v418: 97 pgs: 97 active+clean; 149 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2024-09-19T09:35:21.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:21 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:35:21.299+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:21.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:21 smithi028 ceph-mon[30972]: pgmap v418: 97 pgs: 97 active+clean; 149 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2024-09-19T09:35:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:22 smithi078 ceph-mon[35619]: scrub summary: active paths [/] 2024-09-19T09:35:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:22 smithi078 ceph-mon[35619]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:22 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:35:22.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:22 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:22.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:22 smithi038 ceph-mon[35271]: scrub summary: active paths [/] 2024-09-19T09:35:22.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:22 smithi038 ceph-mon[35271]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:22.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:22 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:35:22.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:22 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:22.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:22 smithi028 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-19T09:35:22.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:22 smithi028 ceph-mon[30972]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T09:35:22.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:22 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] up:active 2024-09-19T09:35:22.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:22 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:35:23.606 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:23 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:23 smithi078 ceph-mon[35619]: pgmap v419: 97 pgs: 97 active+clean; 148 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-09-19T09:35:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:23 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:23 smithi038 ceph-mon[35271]: pgmap v419: 97 pgs: 97 active+clean; 148 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-09-19T09:35:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:23 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:23 smithi028 ceph-mon[30972]: pgmap v419: 97 pgs: 97 active+clean; 148 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-09-19T09:35:25.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:25 smithi078 ceph-mon[35619]: pgmap v420: 97 pgs: 97 active+clean; 144 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 932 op/s 2024-09-19T09:35:25.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:25 smithi038 ceph-mon[35271]: pgmap v420: 97 pgs: 97 active+clean; 144 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 932 op/s 2024-09-19T09:35:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:25 smithi028 ceph-mon[30972]: pgmap v420: 97 pgs: 97 active+clean; 144 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 932 op/s 2024-09-19T09:35:26.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:26 smithi078 ceph-mon[35619]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:26.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:26 smithi038 ceph-mon[35271]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:26.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:26 smithi028 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-19T09:35:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:27 smithi078 ceph-mon[35619]: pgmap v421: 97 pgs: 97 active+clean; 136 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 982 op/s 2024-09-19T09:35:27.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:27 smithi038 ceph-mon[35271]: pgmap v421: 97 pgs: 97 active+clean; 136 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 982 op/s 2024-09-19T09:35:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:27 smithi028 ceph-mon[30972]: pgmap v421: 97 pgs: 97 active+clean; 136 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 982 op/s 2024-09-19T09:35:29.039 INFO:tasks.workunit.client.0.smithi028.stdout: 64 565433 769 373552 845 231636 1259 2024-09-19T09:35:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:29 smithi078 ceph-mon[35619]: pgmap v422: 97 pgs: 97 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-09-19T09:35:29.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:29 smithi038 ceph-mon[35271]: pgmap v422: 97 pgs: 97 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-09-19T09:35:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:29 smithi028 ceph-mon[30972]: pgmap v422: 97 pgs: 97 active+clean; 124 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-09-19T09:35:30.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:30 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:35:30.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:30 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:35:30.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:30 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:35:30.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:30 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:35:30.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:30 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:35:30.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:30 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:35:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:31 smithi078 ceph-mon[35619]: pgmap v423: 97 pgs: 97 active+clean; 130 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.07k op/s 2024-09-19T09:35:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:31 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:35:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:31 smithi078 ceph-mon[35619]: scrub summary: idle 2024-09-19T09:35:31.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:31 smithi038 ceph-mon[35271]: pgmap v423: 97 pgs: 97 active+clean; 130 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.07k op/s 2024-09-19T09:35:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:31 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:35:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:31 smithi038 ceph-mon[35271]: scrub summary: idle 2024-09-19T09:35:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:31 smithi028 ceph-mon[30972]: pgmap v423: 97 pgs: 97 active+clean; 130 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.07k op/s 2024-09-19T09:35:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:31 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:35:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:31 smithi028 ceph-mon[30972]: scrub summary: idle 2024-09-19T09:35:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:33 smithi078 ceph-mon[35619]: pgmap v424: 97 pgs: 97 active+clean; 140 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 1.05k op/s 2024-09-19T09:35:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:33 smithi038 ceph-mon[35271]: pgmap v424: 97 pgs: 97 active+clean; 140 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 1.05k op/s 2024-09-19T09:35:33.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:33 smithi028 ceph-mon[30972]: pgmap v424: 97 pgs: 97 active+clean; 140 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 1.05k op/s 2024-09-19T09:35:33.953 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:35:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fbeffd640 1 -- 172.21.15.28:0/856844954 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00319b0 con 0x7f8fc010b4e0 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 -- 172.21.15.28:0/856844954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc010b4e0 msgr2=0x7f8fc010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 --2- 172.21.15.28:0/856844954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc010b4e0 0x7f8fc010d950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009930 tx=0x7f8fb002a420 comp rx=0 tx=0).stop 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 -- 172.21.15.28:0/856844954 shutdown_connections 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 --2- 172.21.15.28:0/856844954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 0x7f8fc0110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 --2- 172.21.15.28:0/856844954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc010b4e0 0x7f8fc010d950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 --2- 172.21.15.28:0/856844954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc0072d80 0x7f8fc0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 -- 172.21.15.28:0/856844954 >> 172.21.15.28:0/856844954 conn(0x7f8fc006eb10 msgr2=0x7f8fc0070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 -- 172.21.15.28:0/856844954 shutdown_connections 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.735+0000 7f8fc5475640 1 -- 172.21.15.28:0/856844954 wait complete. 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.736+0000 7f8fc5475640 1 Processor -- start 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.736+0000 7f8fc5475640 1 -- start start 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc010b4e0 0x7f8fc01a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 0x7f8fc01ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c430 con 0x7f8fc010de90 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c2b0 con 0x7f8fc010b4e0 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc5475640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c5b0 con 0x7f8fc0072d80 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:42328/0 (socket says 172.21.15.28:42328) 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 -- 172.21.15.28:0/2682467286 learned_addr learned my addr 172.21.15.28:0/2682467286 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc010b4e0 0x7f8fc01a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 0x7f8fc01ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 -- 172.21.15.28:0/2682467286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc010b4e0 msgr2=0x7f8fc01a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc010b4e0 0x7f8fc01a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 -- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 msgr2=0x7f8fc01ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 0x7f8fc01ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.737+0000 7f8fbffff640 1 -- 172.21.15.28:0/2682467286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc01ae700 con 0x7f8fc0072d80 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.738+0000 7f8fbffff640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d950 tx=0x7f8fb400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.738+0000 7f8fbd7fa640 1 -- 172.21.15.28:0/2682467286 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4017070 con 0x7f8fc0072d80 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.738+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01ae9f0 con 0x7f8fc0072d80 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.738+0000 7f8fbd7fa640 1 -- 172.21.15.28:0/2682467286 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb4009c30 con 0x7f8fc0072d80 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.738+0000 7f8fbd7fa640 1 -- 172.21.15.28:0/2682467286 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400f6b0 con 0x7f8fc0072d80 2024-09-19T09:35:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.739+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01aef60 con 0x7f8fc0072d80 2024-09-19T09:35:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.753+0000 7f8fc9472740 1 Processor -- start 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 -- start start 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c1b70 0x555a169c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a169252f0 con 0x555a169c1b70 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a168cd270 con 0x555a169c7540 2024-09-19T09:35:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.754+0000 7f8fc9472740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a169212b0 con 0x555a169c24b0 2024-09-19T09:35:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38594/0 (socket says 172.21.15.28:38594) 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/3952070419 learned_addr learned my addr 172.21.15.28:0/3952070419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c1b70 0x555a169c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/3952070419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 msgr2=0x555a169c6f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169c6f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/3952070419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c1b70 msgr2=0x555a169c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c1b70 0x555a169c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.755+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/3952070419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a168cc340 con 0x555a169c7540 2024-09-19T09:35:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.756+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00314e0 tx=0x7f8fb0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36cb9e460912352 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.756+0000 7f8fbd7fa640 1 -- 172.21.15.28:0/2682467286 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8fb40109d0 con 0x7f8fc0072d80 2024-09-19T09:35:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.758+0000 7f8fbd7fa640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 0x7f8fa80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.758+0000 7f8fbd7fa640 1 -- 172.21.15.28:0/2682467286 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8fb40cedd0 con 0x7f8fc0072d80 2024-09-19T09:35:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.759+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 0x7f8fa80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.760+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 0x7f8fa80a57c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005fd0 tx=0x7f8fb8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.769+0000 7f8fa67fc640 1 -- 172.21.15.28:0/3952070419 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0040070 con 0x555a169c7540 2024-09-19T09:35:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.769+0000 7f8fa67fc640 1 -- 172.21.15.28:0/3952070419 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0004480 con 0x555a169c7540 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 -- 172.21.15.28:0/3952070419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 msgr2=0x555a169c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00314e0 tx=0x7f8fb0005c00 comp rx=0 tx=0).stop 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 -- 172.21.15.28:0/3952070419 shutdown_connections 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169c9a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 --2- 172.21.15.28:0/3952070419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c1b70 0x555a169c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.770+0000 7f8fc9472740 1 -- 172.21.15.28:0/3952070419 >> 172.21.15.28:0/3952070419 conn(0x555a169bd1e0 msgr2=0x555a169bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.776+0000 7f8fc9472740 1 -- 172.21.15.28:0/3952070419 shutdown_connections 2024-09-19T09:35:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.776+0000 7f8fc9472740 1 -- 172.21.15.28:0/3952070419 wait complete. 2024-09-19T09:35:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.776+0000 7f8fc9472740 1 -- start start 2024-09-19T09:35:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 Processor -- start 2024-09-19T09:35:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c1b70 0x555a169dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 0x555a169e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a169db510 con 0x555a169c7540 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a168ae290 con 0x555a169c1b70 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a169db690 con 0x555a169c24b0 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fc9472740 10 client.0 _ll_register_callbacks cb 0x555a168df140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:42340/0 (socket says 172.21.15.28:42340) 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.777+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/4191619945 learned_addr learned my addr 172.21.15.28:0/4191619945 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 0x555a169e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbffff640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c1b70 0x555a169dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/4191619945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c1b70 msgr2=0x555a169dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c1b70 0x555a169dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/4191619945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 msgr2=0x555a169e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 0x555a169e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/4191619945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a16638210 con 0x555a169c24b0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 0x555a169e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0004650 tx=0x7f8fb0004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a870be7c62185ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.778+0000 7f8f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0040070 con 0x555a169c24b0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8fb0004280 con 0x555a169c24b0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb0004480 con 0x555a169c24b0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8fb0031ba0 con 0x555a169c24b0 2024-09-19T09:35:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8fc9472740 10 client.25831 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.779+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555a16704620 con 0x555a169c24b0 2024-09-19T09:35:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.780+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0004480 con 0x555a169c24b0 2024-09-19T09:35:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.780+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f8fb00036a0 con 0x555a169c24b0 2024-09-19T09:35:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 10 client.25831 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 20 client.25831 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a169e8610 0x555a169eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 4 client.25831 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a16932a20 con 0x555a169e8610 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.781+0000 7f8fbffff640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a169e8610 0x555a169eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.782+0000 7f8f9f7fe640 10 client.25831 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.782+0000 7f8fbffff640 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a169e8610 0x555a169eaa00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437256628142abc1 server_cookie=cc08bff5f4c5c0e3 in_seq=0 out_seq=0 2024-09-19T09:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/4191619945 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a16932a20 con 0x555a169e8610 2024-09-19T09:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8f9f7fe640 10 client.25831 handle_command_reply: tid=0 2024-09-19T09:35:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 client.25831 shutdown 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 msgr2=0x555a169dd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0004650 tx=0x7f8fb0004980 comp rx=0 tx=0).stop 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 shutdown_connections 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a169e8610 0x555a169eaa00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c7540 0x555a169e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c24b0 0x555a169dd2d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 --2- 172.21.15.28:0/4191619945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c1b70 0x555a169dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 >> 172.21.15.28:0/4191619945 conn(0x555a169bd1e0 msgr2=0x555a169be9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.786+0000 7f8f9f7fe640 0 client.25831 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.787+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 shutdown_connections 2024-09-19T09:35:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.787+0000 7f8fc9472740 1 -- 172.21.15.28:0/4191619945 wait complete. 2024-09-19T09:35:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.787+0000 7f8fc9472740 20 client.25831 trim_cache size 0 max 16384 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 Processor -- start 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 -- start start 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169beb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01af4d0 con 0x555a169c24b0 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b10d0 con 0x555a169c7540 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.819+0000 7f8fc9472740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01afba0 con 0x555a169c1b70 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33666/0 (socket says 172.21.15.28:33666) 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/2085565905 learned_addr learned my addr 172.21.15.28:0/2085565905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbffff640 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/2085565905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 msgr2=0x555a169be5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169be5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/2085565905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 msgr2=0x555a169beb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169beb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 -- 172.21.15.28:0/2085565905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a169db400 con 0x555a169c24b0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0006010 tx=0x7f8fb0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e740e5a110f997 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fa5ffb640 1 -- 172.21.15.28:0/2085565905 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0040070 con 0x555a169c24b0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fa5ffb640 1 -- 172.21.15.28:0/2085565905 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb005d070 con 0x555a169c24b0 2024-09-19T09:35:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.820+0000 7f8fa5ffb640 1 -- 172.21.15.28:0/2085565905 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00035f0 con 0x555a169c24b0 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 -- 172.21.15.28:0/2085565905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 msgr2=0x555a169e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0006010 tx=0x7f8fb0005b50 comp rx=0 tx=0).stop 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 -- 172.21.15.28:0/2085565905 shutdown_connections 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169beb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e42d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 --2- 172.21.15.28:0/2085565905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169be5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 -- 172.21.15.28:0/2085565905 >> 172.21.15.28:0/2085565905 conn(0x555a169bd1e0 msgr2=0x555a169bc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 -- 172.21.15.28:0/2085565905 shutdown_connections 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.822+0000 7f8fc9472740 1 -- 172.21.15.28:0/2085565905 wait complete. 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.823+0000 7f8fc9472740 1 -- start start 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.823+0000 7f8fc9472740 1 Processor -- start 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:42352/0 (socket says 172.21.15.28:42352) 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33682/0 (socket says 172.21.15.28:33682) 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 -- 172.21.15.28:0/108335911 learned_addr learned my addr 172.21.15.28:0/108335911 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b0940 con 0x555a169c24b0 2024-09-19T09:35:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b0270 con 0x555a169c7540 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01174b0 con 0x555a169c1b70 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 10 client.0 _ll_register_callbacks cb 0x7f8fb0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc4c74640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 -- 172.21.15.28:0/108335911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 msgr2=0x555a169e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 -- 172.21.15.28:0/108335911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 msgr2=0x555a169e1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fbffff640 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a169e30f0 con 0x555a169c1b70 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.824+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb005d070 con 0x555a169c1b70 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8fbffff640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40cc900 tx=0x7f8fb4097040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ff85c9e756776a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8f9f7fe640 10 client.25837 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/108335911 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40b3060 con 0x555a169c1b70 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/108335911 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f8fb4097a90 con 0x555a169c1b70 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/108335911 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8fb4020040 con 0x555a169c1b70 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8fc9472740 10 client.25837 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=33}) -- 0x555a169e4360 con 0x555a169c1b70 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/108335911 <== mon.1 v2:172.21.15.78:3300/0 4 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f8fb40b4250 con 0x555a169c1b70 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.825+0000 7f8fc9472740 10 client.25837 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fc9472740 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fc9472740 20 client.25837 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a16a53710 0x555a16a55b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fc9472740 4 client.25837 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a16a53710 0x555a16a55b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a1692cd70 con 0x555a16a53710 2024-09-19T09:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8f9f7fe640 10 client.25837 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:34.826+0000 7f8fbf7fe640 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a16a53710 0x555a16a55b30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db69309d404c1794 server_cookie=4affb7c608f6c4c3 in_seq=0 out_seq=0 2024-09-19T09:35:34.891 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:35.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.506+0000 7ff08e405640 1 -- 172.21.15.28:0/1183055843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109450 msgr2=0x7ff088110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.506+0000 7ff08e405640 1 --2- 172.21.15.28:0/1183055843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109450 0x7ff088110250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff078009860 tx=0x7ff07802a380 comp rx=0 tx=0).stop 2024-09-19T09:35:35.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.509+0000 7ff08e405640 1 -- 172.21.15.28:0/1183055843 shutdown_connections 2024-09-19T09:35:35.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.509+0000 7ff08e405640 1 --2- 172.21.15.28:0/1183055843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff08811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.509+0000 7ff08e405640 1 --2- 172.21.15.28:0/1183055843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109830 0x7ff088109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.509+0000 7ff08e405640 1 --2- 172.21.15.28:0/1183055843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109450 0x7ff088110250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.509+0000 7ff08e405640 1 -- 172.21.15.28:0/1183055843 >> 172.21.15.28:0/1183055843 conn(0x7ff08806e140 msgr2=0x7ff08806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.512+0000 7ff08e405640 1 -- 172.21.15.28:0/1183055843 shutdown_connections 2024-09-19T09:35:35.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.514+0000 7ff08e405640 1 -- 172.21.15.28:0/1183055843 wait complete. 2024-09-19T09:35:35.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.515+0000 7ff08e405640 1 Processor -- start 2024-09-19T09:35:35.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 -- start start 2024-09-19T09:35:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109450 0x7ff088118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109830 0x7ff088118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff088120e30 con 0x7ff08811b140 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff088120cb0 con 0x7ff088109830 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.519+0000 7ff08e405640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff088120fb0 con 0x7ff088109450 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33692/0 (socket says 172.21.15.28:33692) 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 -- 172.21.15.28:0/3964807019 learned_addr learned my addr 172.21.15.28:0/3964807019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff087fff640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109450 0x7ff088118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 -- 172.21.15.28:0/3964807019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109450 msgr2=0x7ff088118680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109450 0x7ff088118680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 -- 172.21.15.28:0/3964807019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109830 msgr2=0x7ff088118bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109830 0x7ff088118bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.520+0000 7ff08c9a2640 1 -- 172.21.15.28:0/3964807019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088112180 con 0x7ff08811b140 2024-09-19T09:35:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.521+0000 7ff08c9a2640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff08000efc0 tx=0x7ff08000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.521+0000 7ff0857fa640 1 -- 172.21.15.28:0/3964807019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080004b20 con 0x7ff08811b140 2024-09-19T09:35:35.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.521+0000 7ff0857fa640 1 -- 172.21.15.28:0/3964807019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff080003e50 con 0x7ff08811b140 2024-09-19T09:35:35.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.522+0000 7ff0857fa640 1 -- 172.21.15.28:0/3964807019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08000b630 con 0x7ff08811b140 2024-09-19T09:35:35.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.524+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff088112470 con 0x7ff08811b140 2024-09-19T09:35:35.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.525+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881163e0 con 0x7ff08811b140 2024-09-19T09:35:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.530+0000 7ff0907de740 1 Processor -- start 2024-09-19T09:35:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.539+0000 7ff0857fa640 1 -- 172.21.15.28:0/3964807019 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff08001d020 con 0x7ff08811b140 2024-09-19T09:35:35.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 -- start start 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d7dc0 0x55b8607e2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607e2680 0x55b8607e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b86073d2e0 con 0x55b8607d74a0 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b8606e4ce0 con 0x55b8607e2680 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.544+0000 7ff0907de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8607392a0 con 0x55b8607d7dc0 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.545+0000 7ff0857fa640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 0x7ff0700a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.545+0000 7ff0857fa640 1 -- 172.21.15.28:0/3964807019 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff0800cf8a0 con 0x7ff08811b140 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.546+0000 7ff08c9a2640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 0x7ff0700a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.547+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607e2680 0x55b8607e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.547+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.547+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33702/0 (socket says 172.21.15.28:33702) 2024-09-19T09:35:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.547+0000 7ff0877fe640 1 -- 172.21.15.28:0/562978610 learned_addr learned my addr 172.21.15.28:0/562978610 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.549+0000 7ff0877fe640 1 -- 172.21.15.28:0/562978610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d7dc0 msgr2=0x55b8607e2140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.549+0000 7ff0877fe640 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d7dc0 0x55b8607e2140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.549+0000 7ff0877fe640 1 -- 172.21.15.28:0/562978610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607e2680 msgr2=0x55b8607e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.549+0000 7ff0877fe640 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607e2680 0x55b8607e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.549+0000 7ff0877fe640 1 -- 172.21.15.28:0/562978610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8606c9440 con 0x55b8607d74a0 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.551+0000 7ff08c9a2640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 0x7ff0700a5800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff08009be20 tx=0x7ff080004a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.552+0000 7ff0877fe640 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00afc0 tx=0x7ff07c0068d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e50c603b8adba51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.553+0000 7ff06e7fc640 1 -- 172.21.15.28:0/562978610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c011040 con 0x55b8607d74a0 2024-09-19T09:35:35.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.553+0000 7ff06e7fc640 1 -- 172.21.15.28:0/562978610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c004500 con 0x55b8607d74a0 2024-09-19T09:35:35.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.553+0000 7ff06e7fc640 1 -- 172.21.15.28:0/562978610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c004a60 con 0x55b8607d74a0 2024-09-19T09:35:35.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.556+0000 7ff0907de740 1 -- 172.21.15.28:0/562978610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 msgr2=0x55b8607d7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.556+0000 7ff0907de740 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00afc0 tx=0x7ff07c0068d0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 -- 172.21.15.28:0/562978610 shutdown_connections 2024-09-19T09:35:35.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607e2680 0x55b8607e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d7dc0 0x55b8607e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 --2- 172.21.15.28:0/562978610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607d7880 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 -- 172.21.15.28:0/562978610 >> 172.21.15.28:0/562978610 conn(0x55b8607d3a30 msgr2=0x55b8607d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.557+0000 7ff0907de740 1 -- 172.21.15.28:0/562978610 shutdown_connections 2024-09-19T09:35:35.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.558+0000 7ff0907de740 1 -- 172.21.15.28:0/562978610 wait complete. 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.558+0000 7ff0907de740 1 -- start start 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.558+0000 7ff0907de740 1 Processor -- start 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d74a0 0x55b8607e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b8607df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b8607e5250 con 0x55b8607d7dc0 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b8606f8050 con 0x55b8607d74a0 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8607e53d0 con 0x55b8607e2680 2024-09-19T09:35:35.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff0907de740 10 client.0 _ll_register_callbacks cb 0x55b860738160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff087fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff087fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33706/0 (socket says 172.21.15.28:33706) 2024-09-19T09:35:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.559+0000 7ff087fff640 1 -- 172.21.15.28:0/948991788 learned_addr learned my addr 172.21.15.28:0/948991788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 -- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 msgr2=0x55b8607df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b8607df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 -- 172.21.15.28:0/948991788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d74a0 msgr2=0x55b8607e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d74a0 0x55b8607e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 -- 172.21.15.28:0/948991788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8604ae3c0 con 0x55b8607d7dc0 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff087fff640 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a8f0 tx=0x7ff0780095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7991a94eadeb796 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.560+0000 7ff0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.561+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078037040 con 0x55b8607d7dc0 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.561+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07c004500 con 0x55b8607d7dc0 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.561+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff078004030 con 0x55b8607d7dc0 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.561+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff078004230 con 0x55b8607d7dc0 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.562+0000 7ff0907de740 10 client.16293 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.562+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b8607f0c70 con 0x55b8607d7dc0 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c004500 con 0x55b8607d7dc0 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7ff0780049d0 con 0x55b8607d7dc0 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 10 client.16293 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 20 client.16293 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b8607f2630 0x55b8607f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 4 client.16293 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.564+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86074a850 con 0x55b8607f2630 2024-09-19T09:35:35.573 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:35 smithi028 ceph-mon[30972]: pgmap v425: 97 pgs: 97 active+clean; 168 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 1.01k op/s 2024-09-19T09:35:35.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.570+0000 7ff0877fe640 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b8607f2630 0x55b8607f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.576+0000 7ff0877fe640 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b8607f2630 0x55b8607f4ad0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d41dee6aa1d10a0 server_cookie=5897121ce9f174da in_seq=0 out_seq=0 2024-09-19T09:35:35.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.576+0000 7ff0677fe640 10 client.16293 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:35.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.584+0000 7ff0677fe640 1 -- 172.21.15.28:0/948991788 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b86074a850 con 0x55b8607f2630 2024-09-19T09:35:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.584+0000 7ff0677fe640 10 client.16293 handle_command_reply: tid=0 2024-09-19T09:35:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.584+0000 7ff0907de740 1 client.16293 shutdown 2024-09-19T09:35:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.585+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 msgr2=0x55b8607df290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.585+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a8f0 tx=0x7ff0780095a0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 shutdown_connections 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b8607f2630 0x55b8607f4ad0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b8607df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d7dc0 0x55b8607df290 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 --2- 172.21.15.28:0/948991788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d74a0 0x55b8607e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0677fe640 0 client.16293 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 >> 172.21.15.28:0/948991788 conn(0x55b8607d3a30 msgr2=0x55b8607d9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.586+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 shutdown_connections 2024-09-19T09:35:35.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.588+0000 7ff0907de740 1 -- 172.21.15.28:0/948991788 wait complete. 2024-09-19T09:35:35.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.588+0000 7ff0907de740 20 client.16293 trim_cache size 0 max 16384 2024-09-19T09:35:35.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.629+0000 7ff0907de740 1 Processor -- start 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.631+0000 7ff0907de740 1 -- start start 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.631+0000 7ff0907de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 0x55b8607deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff0907de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff0907de740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607e2680 0x55b8607fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff087fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff087fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38650/0 (socket says 172.21.15.28:38650) 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff087fff640 1 -- 172.21.15.28:0/378230818 learned_addr learned my addr 172.21.15.28:0/378230818 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c4980 con 0x55b8607e2680 2024-09-19T09:35:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c6580 con 0x55b8607d7dc0 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.632+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c5050 con 0x55b8607d74a0 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 0x55b8607deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 -- 172.21.15.28:0/378230818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 msgr2=0x55b8607e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 -- 172.21.15.28:0/378230818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607e2680 msgr2=0x55b8607fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607e2680 0x55b8607fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.633+0000 7ff0877fe640 1 -- 172.21.15.28:0/378230818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8607d7bf0 con 0x55b8607d74a0 2024-09-19T09:35:35.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.634+0000 7ff0877fe640 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 0x55b8607deb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b8607f5310 tx=0x7ff07c00a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b7910b4e4b934f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.635+0000 7ff06dffb640 1 -- 172.21.15.28:0/378230818 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c027040 con 0x55b8607d74a0 2024-09-19T09:35:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.635+0000 7ff087fff640 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.635+0000 7ff06dffb640 1 -- 172.21.15.28:0/378230818 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c00ac60 con 0x55b8607d74a0 2024-09-19T09:35:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.635+0000 7ff06dffb640 1 -- 172.21.15.28:0/378230818 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c01d040 con 0x55b8607d74a0 2024-09-19T09:35:35.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.636+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 msgr2=0x55b8607deb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.636+0000 7ff0907de740 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 0x55b8607deb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55b8607f5310 tx=0x7ff07c00a7f0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.638+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 shutdown_connections 2024-09-19T09:35:35.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.638+0000 7ff0907de740 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.638+0000 7ff0907de740 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607e2680 0x55b8607fa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.638+0000 7ff0907de740 1 --2- 172.21.15.28:0/378230818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607d74a0 0x55b8607deb50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.639+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 >> 172.21.15.28:0/378230818 conn(0x55b8607d3a30 msgr2=0x55b8607e0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.641+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 shutdown_connections 2024-09-19T09:35:35.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.641+0000 7ff0907de740 1 -- 172.21.15.28:0/378230818 wait complete. 2024-09-19T09:35:35.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.642+0000 7ff0907de740 1 -- start start 2024-09-19T09:35:35.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.642+0000 7ff0907de740 1 Processor -- start 2024-09-19T09:35:35.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.643+0000 7ff0907de740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.643+0000 7ff0907de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.643+0000 7ff0907de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b860854330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.643+0000 7ff087fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.643+0000 7ff087fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38664/0 (socket says 172.21.15.28:38664) 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff087fff640 1 -- 172.21.15.28:0/1165928480 learned_addr learned my addr 172.21.15.28:0/1165928480 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff0877fe640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b860854330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c5df0 con 0x55b8607d74a0 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c5720 con 0x55b8607d7dc0 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881c2960 con 0x55b8607e2680 2024-09-19T09:35:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.644+0000 7ff0907de740 10 client.0 _ll_register_callbacks cb 0x7ff078035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:35.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.646+0000 7ff0877fe640 1 -- 172.21.15.28:0/1165928480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 msgr2=0x55b8607e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.646+0000 7ff0877fe640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.646+0000 7ff0877fe640 1 -- 172.21.15.28:0/1165928480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 msgr2=0x55b8607e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.646+0000 7ff0877fe640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.646+0000 7ff0877fe640 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b860854970 con 0x55b8607e2680 2024-09-19T09:35:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.647+0000 7ff0877fe640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b860854330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55b8607f5310 tx=0x7ff07c00aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38be4e9fc93f57c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.647+0000 7ff0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:35.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.648+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c027070 con 0x55b8607e2680 2024-09-19T09:35:35.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.648+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07c00ac60 con 0x55b8607e2680 2024-09-19T09:35:35.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.649+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff07c0036b0 con 0x55b8607e2680 2024-09-19T09:35:35.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.649+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff07c0063b0 con 0x55b8607e2680 2024-09-19T09:35:35.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.651+0000 7ff0907de740 10 client.25846 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:35.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.651+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b86085cea0 con 0x55b8607e2680 2024-09-19T09:35:35.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.653+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c00ac60 con 0x55b8607e2680 2024-09-19T09:35:35.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.653+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7ff07c038a60 con 0x55b8607e2680 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 10 client.25846 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 20 client.25846 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b86085d370 0x55b86085f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 4 client.25846 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:35:35.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.654+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b860744d60 con 0x55b86085d370 2024-09-19T09:35:35.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.657+0000 7ff087fff640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b86085d370 0x55b86085f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:35.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.658+0000 7ff0677fe640 10 client.25846 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:35.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.658+0000 7ff087fff640 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b86085d370 0x55b86085f790 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140c645622ed5913 server_cookie=2f549d7a7b0be8c2 in_seq=0 out_seq=0 2024-09-19T09:35:35.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.664+0000 7ff0677fe640 1 -- 172.21.15.28:0/1165928480 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b860744d60 con 0x55b86085d370 2024-09-19T09:35:35.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.664+0000 7ff0677fe640 10 client.25846 handle_command_reply: tid=0 2024-09-19T09:35:35.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.664+0000 7ff0907de740 1 client.25846 shutdown 2024-09-19T09:35:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 msgr2=0x55b860854330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b860854330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55b8607f5310 tx=0x7ff07c00aea0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 shutdown_connections 2024-09-19T09:35:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b8607d7dc0 0x55b8607e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b86085d370 0x55b86085f790 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8607e2680 0x55b860854330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 --2- 172.21.15.28:0/1165928480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b8607d74a0 0x55b8607e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 >> 172.21.15.28:0/1165928480 conn(0x55b8607d3a30 msgr2=0x55b8607d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0677fe640 0 client.25846 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 shutdown_connections 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.669+0000 7ff0907de740 1 -- 172.21.15.28:0/1165928480 wait complete. 2024-09-19T09:35:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.670+0000 7ff0907de740 20 client.25846 trim_cache size 0 max 16384 2024-09-19T09:35:35.672 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:35:35.672 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2024-09-19T09:35:35.672 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2024-09-19T09:35:35.672 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 msgr2=0x7ff0700a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 0x7ff0700a5800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff08009be20 tx=0x7ff080004a00 comp rx=0 tx=0).stop 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 msgr2=0x7ff0881118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff08000efc0 tx=0x7ff08000c800 comp rx=0 tx=0).stop 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 shutdown_connections 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff0700a3340 0x7ff0700a5800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088109830 0x7ff088118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff08811b140 0x7ff0881118c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 --2- 172.21.15.28:0/3964807019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff088109450 0x7ff088118680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.671+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 >> 172.21.15.28:0/3964807019 conn(0x7ff08806e140 msgr2=0x7ff08811beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.672+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 shutdown_connections 2024-09-19T09:35:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:35.672+0000 7ff08e405640 1 -- 172.21.15.28:0/3964807019 wait complete. 2024-09-19T09:35:35.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:35 smithi038 ceph-mon[35271]: pgmap v425: 97 pgs: 97 active+clean; 168 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 1.01k op/s 2024-09-19T09:35:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:35 smithi078 ceph-mon[35619]: pgmap v425: 97 pgs: 97 active+clean; 168 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 1.01k op/s 2024-09-19T09:35:36.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:35:36.621 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:35:36.899 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:37.621 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:37 smithi028 ceph-mon[30972]: pgmap v426: 97 pgs: 97 active+clean; 188 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 780 op/s 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 -- 172.21.15.28:0/1176513407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 msgr2=0x7f3a0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 --2- 172.21.15.28:0/1176513407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a0010da40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009ca0 tx=0x7f39f802a950 comp rx=0 tx=0).stop 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 -- 172.21.15.28:0/1176513407 shutdown_connections 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 --2- 172.21.15.28:0/1176513407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00102570 0x7f3a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 --2- 172.21.15.28:0/1176513407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a0010da40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.626+0000 7f3a05a28640 1 --2- 172.21.15.28:0/1176513407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00100e20 0x7f3a00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.627+0000 7f3a05a28640 1 -- 172.21.15.28:0/1176513407 >> 172.21.15.28:0/1176513407 conn(0x7f3a000fc7d0 msgr2=0x7f3a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.627+0000 7f3a05a28640 1 -- 172.21.15.28:0/1176513407 shutdown_connections 2024-09-19T09:35:37.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.627+0000 7f3a05a28640 1 -- 172.21.15.28:0/1176513407 wait complete. 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.627+0000 7f3a05a28640 1 Processor -- start 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.627+0000 7f3a05a28640 1 -- start start 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00102570 0x7f3a00111750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a00111c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00110fd0 con 0x7f3a00102ef0 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00110e50 con 0x7f3a00100e20 2024-09-19T09:35:37.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f3a05a28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00111150 con 0x7f3a00102570 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39feffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39feffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38684/0 (socket says 172.21.15.28:38684) 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39feffd640 1 -- 172.21.15.28:0/970234441 learned_addr learned my addr 172.21.15.28:0/970234441 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a00111c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39f7fff640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00102570 0x7f3a00111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 -- 172.21.15.28:0/970234441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00102570 msgr2=0x7f3a00111750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00102570 0x7f3a00111750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 -- 172.21.15.28:0/970234441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 msgr2=0x7f3a00113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39ff7fe640 1 -- 172.21.15.28:0/970234441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a00112550 con 0x7f3a00102ef0 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.628+0000 7f39feffd640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f39ff7fe640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a00111c90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f39f8005e30 tx=0x7f39f8003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:37.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8034040 con 0x7f3a00102ef0 2024-09-19T09:35:37.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f802f930 con 0x7f3a00102ef0 2024-09-19T09:35:37.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a001a6bb0 con 0x7f3a00102ef0 2024-09-19T09:35:37.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80045d0 con 0x7f3a00102ef0 2024-09-19T09:35:37.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.629+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a001a70b0 con 0x7f3a00102ef0 2024-09-19T09:35:37.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.630+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a00101200 con 0x7f3a00102ef0 2024-09-19T09:35:37.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.632+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f39f8037020 con 0x7f3a00102ef0 2024-09-19T09:35:37.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.634+0000 7f3a04a26640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 0x7f39ec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:37.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.634+0000 7f39feffd640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 0x7f39ec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:37.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.634+0000 7f39feffd640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 0x7f39ec0a5780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39f0011500 tx=0x7f39f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:37.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.634+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f39f80f06d0 con 0x7f3a00102ef0 2024-09-19T09:35:37.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.638+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39f8081210 con 0x7f3a00102ef0 2024-09-19T09:35:37.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:37 smithi038 ceph-mon[35271]: pgmap v426: 97 pgs: 97 active+clean; 188 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 780 op/s 2024-09-19T09:35:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:37 smithi078 ceph-mon[35619]: pgmap v426: 97 pgs: 97 active+clean; 188 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 780 op/s 2024-09-19T09:35:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.858+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a001128c0 con 0x7f3a00102ef0 2024-09-19T09:35:37.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.858+0000 7f3a04a26640 1 -- 172.21.15.28:0/970234441 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11957 (secure 0 0 0) 0x7f3a001128c0 con 0x7f3a00102ef0 2024-09-19T09:35:37.860 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:35:37.860 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"btime":"2024-09-19T09:35:21:300725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":25804,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:35:21.216576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:35:37.860 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 msgr2=0x7f39ec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 0x7f39ec0a5780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39f0011500 tx=0x7f39f0005f90 comp rx=0 tx=0).stop 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 msgr2=0x7f3a00111c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a00111c90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f39f8005e30 tx=0x7f39f8003b80 comp rx=0 tx=0).stop 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 shutdown_connections 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f39ec0a32c0 0x7f39ec0a5780 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a00102ef0 0x7f3a00111c90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00102570 0x7f3a00111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 --2- 172.21.15.28:0/970234441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a00100e20 0x7f3a00113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:37.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 >> 172.21.15.28:0/970234441 conn(0x7f3a000fc7d0 msgr2=0x7f3a00103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:37.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 shutdown_connections 2024-09-19T09:35:37.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:37.861+0000 7f3a05a28640 1 -- 172.21.15.28:0/970234441 wait complete. 2024-09-19T09:35:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.036+0000 7f8f9f7fe640 1 -- 172.21.15.28:0/108335911 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a1692cd70 con 0x555a16a53710 2024-09-19T09:35:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.036+0000 7f8f9f7fe640 10 client.25837 handle_command_reply: tid=0 2024-09-19T09:35:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.036+0000 7f8fc9472740 1 client.25837 shutdown 2024-09-19T09:35:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 msgr2=0x555a169e15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40cc900 tx=0x7f8fb4097040 comp rx=0 tx=0).stop 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 shutdown_connections 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a169c7540 0x555a169e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a16a53710 0x555a16a55b30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a169c24b0 0x555a169e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 --2- 172.21.15.28:0/108335911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a169c1b70 0x555a169e15e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 >> 172.21.15.28:0/108335911 conn(0x555a169bd1e0 msgr2=0x555a169bc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8f9f7fe640 0 client.25837 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 shutdown_connections 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 1 -- 172.21.15.28:0/108335911 wait complete. 2024-09-19T09:35:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.037+0000 7f8fc9472740 20 client.25837 trim_cache size 0 max 16384 2024-09-19T09:35:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:35:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:35:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:35:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:35:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 msgr2=0x7f8fa80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 0x7f8fa80a57c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005fd0 tx=0x7f8fb8009600 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 msgr2=0x7f8fc01a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d950 tx=0x7f8fb400de10 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 shutdown_connections 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fa80a3300 0x7f8fa80a57c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc010b4e0 0x7f8fc01a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fc010de90 0x7f8fc01ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 --2- 172.21.15.28:0/2682467286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fc0072d80 0x7f8fc01a6ae0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 >> 172.21.15.28:0/2682467286 conn(0x7f8fc006eb10 msgr2=0x7f8fc010f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 shutdown_connections 2024-09-19T09:35:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:38.038+0000 7f8fc5475640 1 -- 172.21.15.28:0/2682467286 wait complete. 2024-09-19T09:35:38.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:38 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/970234441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:38.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:38 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/970234441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:38.789 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:35:38.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:38 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/970234441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:39.038 INFO:tasks.workunit.client.0.smithi028.stdout: 70 1015926 263 675218 224 432550 844 2024-09-19T09:35:39.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:39 smithi028 ceph-mon[30972]: pgmap v427: 97 pgs: 97 active+clean; 210 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 704 op/s 2024-09-19T09:35:39.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:39 smithi038 ceph-mon[35271]: pgmap v427: 97 pgs: 97 active+clean; 210 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 704 op/s 2024-09-19T09:35:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:39 smithi078 ceph-mon[35619]: pgmap v427: 97 pgs: 97 active+clean; 210 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 704 op/s 2024-09-19T09:35:40.242 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 -- 172.21.15.28:0/478350362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102570 msgr2=0x7fedd41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 --2- 172.21.15.28:0/478350362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102570 0x7fedd41029b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009900 tx=0x7fedc802a480 comp rx=0 tx=0).stop 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 -- 172.21.15.28:0/478350362 shutdown_connections 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 --2- 172.21.15.28:0/478350362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102ef0 0x7fedd410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 --2- 172.21.15.28:0/478350362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102570 0x7fedd41029b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 --2- 172.21.15.28:0/478350362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 -- 172.21.15.28:0/478350362 >> 172.21.15.28:0/478350362 conn(0x7fedd40fc7d0 msgr2=0x7fedd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 -- 172.21.15.28:0/478350362 shutdown_connections 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 -- 172.21.15.28:0/478350362 wait complete. 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.859+0000 7feddb55c640 1 Processor -- start 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 -- start start 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102570 0x7fedd4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 0x7fedd41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110f80 con 0x7fedd4102ef0 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110e00 con 0x7fedd4100e20 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7feddb55c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4111100 con 0x7fedd4102570 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 0x7fedd41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd8af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102570 0x7fedd4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38700/0 (socket says 172.21.15.28:38700) 2024-09-19T09:35:40.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 -- 172.21.15.28:0/1386233897 learned_addr learned my addr 172.21.15.28:0/1386233897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 -- 172.21.15.28:0/1386233897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102570 msgr2=0x7fedd4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102570 0x7fedd4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 -- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 msgr2=0x7fedd41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 0x7fedd41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 -- 172.21.15.28:0/1386233897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a48d0 con 0x7fedd4100e20 2024-09-19T09:35:40.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 0x7fedd41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:40.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.860+0000 7fedd92f8640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedc400ebc0 tx=0x7fedc400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.861+0000 7fedd27fc640 1 -- 172.21.15.28:0/1386233897 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4010630 con 0x7fedd4100e20 2024-09-19T09:35:40.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.861+0000 7fedd27fc640 1 -- 172.21.15.28:0/1386233897 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4010cb0 con 0x7fedd4100e20 2024-09-19T09:35:40.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.861+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a4aa0 con 0x7fedd4100e20 2024-09-19T09:35:40.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.861+0000 7fedd27fc640 1 -- 172.21.15.28:0/1386233897 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4018bd0 con 0x7fedd4100e20 2024-09-19T09:35:40.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.861+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a4f60 con 0x7fedd4100e20 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.863+0000 7fedd27fc640 1 -- 172.21.15.28:0/1386233897 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fedc40107d0 con 0x7fedd4100e20 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.863+0000 7feddd935740 1 Processor -- start 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 -- start start 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dced8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dcee3370 0x5566dcee5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5566dce3e2f0 con 0x5566dced9b40 2024-09-19T09:35:40.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5566dcde5cf0 con 0x5566dcee3370 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7feddd935740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566dce3a2b0 con 0x5566dced85d0 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dced8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33754/0 (socket says 172.21.15.28:33754) 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 -- 172.21.15.28:0/1554210930 learned_addr learned my addr 172.21.15.28:0/1554210930 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd27fc640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 0x7fedbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd8af7640 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dcee3370 0x5566dcee5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 0x7fedbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 -- 172.21.15.28:0/1554210930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 msgr2=0x5566dced8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dced8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 -- 172.21.15.28:0/1554210930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dcee3370 msgr2=0x5566dcee5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dcee3370 0x5566dcee5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.864+0000 7fedd9af9640 1 -- 172.21.15.28:0/1554210930 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566dcdca3a0 con 0x5566dced9b40 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedd27fc640 1 -- 172.21.15.28:0/1386233897 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fedc40cff00 con 0x7fedd4100e20 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0058c0 tx=0x7fedcc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb17aebab34606c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedd9af9640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 0x7fedbc0a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedcc024000 tx=0x7fedcc022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedbb7fe640 1 -- 172.21.15.28:0/1554210930 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc023020 con 0x5566dced9b40 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedbb7fe640 1 -- 172.21.15.28:0/1554210930 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0135a0 con 0x5566dced9b40 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.865+0000 7fedbb7fe640 1 -- 172.21.15.28:0/1554210930 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc027730 con 0x5566dced9b40 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- 172.21.15.28:0/1554210930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 msgr2=0x5566dced8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0058c0 tx=0x7fedcc00f790 comp rx=0 tx=0).stop 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- 172.21.15.28:0/1554210930 shutdown_connections 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dcee3370 0x5566dcee5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dced8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 --2- 172.21.15.28:0/1554210930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dced8090 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- 172.21.15.28:0/1554210930 >> 172.21.15.28:0/1554210930 conn(0x5566dcea1310 msgr2=0x5566dced4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- 172.21.15.28:0/1554210930 shutdown_connections 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- 172.21.15.28:0/1554210930 wait complete. 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 -- start start 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.866+0000 7feddd935740 1 Processor -- start 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced9b40 0x5566dcee0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5566dcee6070 con 0x5566dcee3370 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5566dcdf9060 con 0x5566dced9b40 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566dcee61f0 con 0x5566dced85d0 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7feddd935740 10 client.0 _ll_register_callbacks cb 0x5566dcb4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:40.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:42444/0 (socket says 172.21.15.28:42444) 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 -- 172.21.15.28:0/2025544637 learned_addr learned my addr 172.21.15.28:0/2025544637 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33756/0 (socket says 172.21.15.28:33756) 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd8af7640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced9b40 0x5566dcee0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 -- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced9b40 msgr2=0x5566dcee0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced9b40 0x5566dcee0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 -- 172.21.15.28:0/2025544637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 msgr2=0x5566dcee06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 -- 172.21.15.28:0/2025544637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566dcbaf3c0 con 0x5566dced85d0 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd9af9640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:40.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedd92f8640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedc4098fd0 tx=0x7fedc40d35d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eae4562cbe381ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.867+0000 7fedb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4010e60 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fedc409ba50 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc0135a0 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fedc4098040 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 10 client.25864 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566dcef1bf0 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0135a0 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fedc409b4c0 con 0x5566dced85d0 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 10 client.25864 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 10 client.25864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 20 client.25864 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:40.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcef53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 4 client.25864 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566dce4b860 con 0x5566dcef3000 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.868+0000 7fedd8af7640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcef53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.869+0000 7fedd8af7640 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcef53f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e879d09188f5028 server_cookie=1b19d8cf8f6b37ee in_seq=0 out_seq=0 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.869+0000 7fedb8ff9640 10 client.25864 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.869+0000 7fedb8ff9640 1 -- 172.21.15.28:0/2025544637 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566dce4b860 con 0x5566dcef3000 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.869+0000 7fedb8ff9640 10 client.25864 handle_command_reply: tid=0 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.869+0000 7feddd935740 1 client.25864 shutdown 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 msgr2=0x5566dcee2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedc4098fd0 tx=0x7fedc40d35d0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 shutdown_connections 2024-09-19T09:35:40.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcef53f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced9b40 0x5566dcee0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcee06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7fedb8ff9640 0 client.25864 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 --2- 172.21.15.28:0/2025544637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced85d0 0x5566dcee2d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 >> 172.21.15.28:0/2025544637 conn(0x5566dcea1310 msgr2=0x5566dceda960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 shutdown_connections 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 1 -- 172.21.15.28:0/2025544637 wait complete. 2024-09-19T09:35:40.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.870+0000 7feddd935740 20 client.25864 trim_cache size 0 max 16384 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.883+0000 7feddd935740 1 Processor -- start 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.883+0000 7feddd935740 1 -- start start 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcedf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced9b40 0x5566dceef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41ad410 con 0x5566dcee3370 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41af010 con 0x5566dced85d0 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7feddd935740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41adae0 con 0x5566dced9b40 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33772/0 (socket says 172.21.15.28:33772) 2024-09-19T09:35:40.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 -- 172.21.15.28:0/3327732032 learned_addr learned my addr 172.21.15.28:0/3327732032 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd8af7640 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced9b40 0x5566dceef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd92f8640 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcedf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 -- 172.21.15.28:0/3327732032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced9b40 msgr2=0x5566dceef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced9b40 0x5566dceef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 -- 172.21.15.28:0/3327732032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 msgr2=0x5566dcedf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcedf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 -- 172.21.15.28:0/3327732032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566dcee0f80 con 0x5566dcee3370 2024-09-19T09:35:40.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedd9af9640 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fedcc01fe70 tx=0x7fedcc01eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25e99ca180e2a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.884+0000 7fedbaffd640 1 -- 172.21.15.28:0/3327732032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc023040 con 0x5566dcee3370 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7fedbaffd640 1 -- 172.21.15.28:0/3327732032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc02c020 con 0x5566dcee3370 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7fedbaffd640 1 -- 172.21.15.28:0/3327732032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0052b0 con 0x5566dcee3370 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 -- 172.21.15.28:0/3327732032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 msgr2=0x5566dcefac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fedcc01fe70 tx=0x7fedcc01eea0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 -- 172.21.15.28:0/3327732032 shutdown_connections 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dcee3370 0x5566dcefac20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dced9b40 0x5566dceef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 --2- 172.21.15.28:0/3327732032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcedf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 -- 172.21.15.28:0/3327732032 >> 172.21.15.28:0/3327732032 conn(0x5566dcea1310 msgr2=0x5566dcee20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 -- 172.21.15.28:0/3327732032 shutdown_connections 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.885+0000 7feddd935740 1 -- 172.21.15.28:0/3327732032 wait complete. 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 -- start start 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 Processor -- start 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dcee7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 0x5566dcf5a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41ae880 con 0x5566dced9b40 2024-09-19T09:35:40.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41ae1b0 con 0x5566dced85d0 2024-09-19T09:35:40.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41ab3f0 con 0x5566dcee3370 2024-09-19T09:35:40.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7feddd935740 10 client.0 _ll_register_callbacks cb 0x7fedc40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:40.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7fedd8af7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dcee7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7fedd92f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38726/0 (socket says 172.21.15.28:38726) 2024-09-19T09:35:40.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7fedd92f8640 1 -- 172.21.15.28:0/323902298 learned_addr learned my addr 172.21.15.28:0/323902298 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:40.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.886+0000 7fedd9af9640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 0x5566dcf5a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 -- 172.21.15.28:0/323902298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 msgr2=0x5566dcf5a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 0x5566dcf5a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 -- 172.21.15.28:0/323902298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 msgr2=0x5566dcee7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:40.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dcee7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 -- 172.21.15.28:0/323902298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566dcf5a620 con 0x5566dced85d0 2024-09-19T09:35:40.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd8af7640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dcee7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:40.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd9af9640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 0x5566dcf5a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:40.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedd92f8640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedc4098fd0 tx=0x7fedc40982a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c0104c07dd59f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:40.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:35:40.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4010e60 con 0x5566dced85d0 2024-09-19T09:35:40.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fedc409ba50 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc02c020 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fedc409bf60 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7feddd935740 10 client.25913 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566dcf619a0 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.887+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4010ac0 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fedc40a1bb0 con 0x5566dced85d0 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 10 client.25913 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 10 client.25913 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 20 client.25913 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:40.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcf63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 4 client.25913 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 --> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fedcc02c020 con 0x5566dcef3000 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7fedd8af7640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcf63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7fedd8af7640 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcf63f30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be36104aa5f9e6fb server_cookie=a7bece73a0cf8124 in_seq=0 out_seq=0 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.888+0000 7fedb8ff9640 10 client.25913 ms_handle_connect on v2:172.21.15.38:6836/560457008 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7fedb8ff9640 1 -- 172.21.15.28:0/323902298 <== mds.4 v2:172.21.15.38:6836/560457008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+21340 (crc 0 0 0) 0x7fedcc02c020 con 0x5566dcef3000 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7fedb8ff9640 10 client.25913 handle_command_reply: tid=0 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 client.25913 shutdown 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 msgr2=0x5566dcee6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedc4098fd0 tx=0x7fedc40982a0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 shutdown_connections 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:6836/560457008,v1:172.21.15.38:6837/560457008] conn(0x5566dcef3000 0x5566dcf63f30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566dcee3370 0x5566dcf5a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5566dced9b40 0x5566dcee7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 --2- 172.21.15.28:0/323902298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5566dced85d0 0x5566dcee6bc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 >> 172.21.15.28:0/323902298 conn(0x5566dcea1310 msgr2=0x5566dced34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7fedb8ff9640 0 client.25913 ms_handle_reset on v2:172.21.15.38:6836/560457008 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 shutdown_connections 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 1 -- 172.21.15.28:0/323902298 wait complete. 2024-09-19T09:35:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.889+0000 7feddd935740 20 client.25913 trim_cache size 0 max 16384 2024-09-19T09:35:40.900 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-19T09:35:40.900 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.900 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.901 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 43505211, 2024-09-19T09:35:40.901 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635020, 2024-09-19T09:35:40.901 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml" 2024-09-19T09:35:40.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 61911944, 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635273, 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg" 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.902 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 115109522, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635268, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg" 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 183892634, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635078, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml" 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.903 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 240764108, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635258, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml" 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 338749725, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635328, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg" 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.904 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 400686254, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635062, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml" 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 446759179, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635035, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml" 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.905 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 462664256, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635357, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg" 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 479171130, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635248, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml" 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.906 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 504655425, 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635322, 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg" 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 513296571, 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635042, 2024-09-19T09:35:40.907 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml" 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 549348820, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635399, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg" 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 755790205, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635098, 2024-09-19T09:35:40.908 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml" 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 761139812, 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635017, 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml" 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 762961691, 2024-09-19T09:35:40.909 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635058, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml" 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 777647174, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635014, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml" 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 789521767, 2024-09-19T09:35:40.910 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635083, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml" 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 822673586, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635038, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml" 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 826824813, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635361, 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg" 2024-09-19T09:35:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 831921796, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635010, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml" 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 865945515, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635263, 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml" 2024-09-19T09:35:40.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 866826736, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635103, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml" 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 916280144, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635337, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg" 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 948788196, 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635395, 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg" 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 972844044, 2024-09-19T09:35:40.914 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635316, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg" 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 995828104, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635093, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml" 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1048844883, 2024-09-19T09:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635050, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml" 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1078908937, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635310, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg" 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1110953659, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635133, 2024-09-19T09:35:40.916 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml" 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1111906343, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635238, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml" 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1154068137, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635046, 2024-09-19T09:35:40.917 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml" 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1160714455, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635158, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml" 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1181845281, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635193, 2024-09-19T09:35:40.918 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml" 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1245952906, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635029, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml" 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1262330276, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635183, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml" 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1273027543, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635218, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml" 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1296281627, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635148, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml" 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.920 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1478194775, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635379, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg" 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1478222328, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635001, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml" 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.921 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1486367090, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635253, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml" 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1532872448, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635143, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml" 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1545842105, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635384, 2024-09-19T09:35:40.922 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg" 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1549326968, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635341, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg" 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1570516012, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635304, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg" 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1589658122, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635208, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml" 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1593190922, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635163, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml" 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.924 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1594183362, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635298, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg" 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1776254104, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635333, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg" 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1778768791, 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635032, 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml" 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1814649507, 2024-09-19T09:35:40.926 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635198, 2024-09-19T09:35:40.927 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml" 2024-09-19T09:35:40.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.927 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.927 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.927 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1953341074, 2024-09-19T09:35:40.928 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635074, 2024-09-19T09:35:40.928 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml" 2024-09-19T09:35:40.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.928 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.928 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2079664567, 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635233, 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml" 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.929 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2095444442, 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635004, 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml" 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2142132056, 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635123, 2024-09-19T09:35:40.930 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml" 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2335563959, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635203, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml" 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2440389343, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635228, 2024-09-19T09:35:40.931 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml" 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2462381569, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635369, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg" 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2473727998, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635153, 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml" 2024-09-19T09:35:40.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2527562726, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635128, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml" 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2620378249, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635345, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg" 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.933 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2802340230, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635108, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml" 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2838364705, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635173, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml" 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.934 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2918815820, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635188, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml" 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2941740507, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635292, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg" 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2968186054, 2024-09-19T09:35:40.935 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635374, 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg" 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2993810965, 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635286, 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg" 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.936 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3036187249, 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635113, 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 msgr2=0x7fedbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 0x7fedbc0a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedcc024000 tx=0x7fedcc022000 comp rx=0 tx=0).stop 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 msgr2=0x7fedd4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedc400ebc0 tx=0x7fedc400c6a0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 shutdown_connections 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fedbc0a3240 0x7fedbc0a5700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedd4102ef0 0x7fedd41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fedd4102570 0x7fedd4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 --2- 172.21.15.28:0/1386233897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedd4100e20 0x7fedd4199310 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 >> 172.21.15.28:0/1386233897 conn(0x7fedd40fc7d0 msgr2=0x7fedd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 shutdown_connections 2024-09-19T09:35:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:40.890+0000 7feddb55c640 1 -- 172.21.15.28:0/1386233897 wait complete. 2024-09-19T09:35:40.938 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml" 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3209832212, 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635088, 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml" 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3243746793, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635178, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml" 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3421895188, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635070, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml" 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.940 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3463495752, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511634999, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml" 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3509476447, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635026, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml" 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3526153903, 2024-09-19T09:35:40.941 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635390, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg" 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3531836474, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635280, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg" 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3542167398, 2024-09-19T09:35:40.942 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635365, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg" 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3673562544, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635353, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg" 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3703137601, 2024-09-19T09:35:40.943 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635066, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml" 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3758349876, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635213, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml" 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3778975836, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635023, 2024-09-19T09:35:40.944 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml" 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3909603654, 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635138, 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml" 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.945 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3930130926, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635054, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml" 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3937188258, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635223, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml" 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3972249198, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635168, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml" 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3993397751, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635118, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml" 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4094002322, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635243, 2024-09-19T09:35:40.947 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml" 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4174800544, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635008, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml" 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4290209847, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635349, 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg" 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:35:40.948 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-19T09:35:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:41 smithi038 ceph-mon[35271]: pgmap v428: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T09:35:41.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:41 smithi028 ceph-mon[30972]: pgmap v428: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T09:35:41.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:41 smithi078 ceph-mon[35619]: pgmap v428: 97 pgs: 97 active+clean; 233 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-19T09:35:42.465 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:35:43.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:43 smithi038 ceph-mon[35271]: pgmap v429: 97 pgs: 97 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T09:35:43.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:43 smithi028 ceph-mon[30972]: pgmap v429: 97 pgs: 97 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T09:35:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:43 smithi078 ceph-mon[35619]: pgmap v429: 97 pgs: 97 active+clean; 232 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2024-09-19T09:35:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:45 smithi038 ceph-mon[35271]: pgmap v430: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.46k op/s 2024-09-19T09:35:45.677 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:45 smithi028 ceph-mon[30972]: pgmap v430: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.46k op/s 2024-09-19T09:35:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:45 smithi078 ceph-mon[35619]: pgmap v430: 97 pgs: 97 active+clean; 210 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.46k op/s 2024-09-19T09:35:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.897+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1866851214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 msgr2=0x7f7dd4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.897+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1866851214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4107ba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009930 tx=0x7f7dc802a420 comp rx=0 tx=0).stop 2024-09-19T09:35:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1866851214 shutdown_connections 2024-09-19T09:35:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1866851214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd41082e0 0x7f7dd410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1866851214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4107ba0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1866851214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd4102db0 0x7f7dd4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1866851214 >> 172.21.15.28:0/1866851214 conn(0x7f7dd40fc7d0 msgr2=0x7f7dd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1866851214 shutdown_connections 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1866851214 wait complete. 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 Processor -- start 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.898+0000 7f7dd9ce1640 1 -- start start 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 0x7f7dd41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 0x7f7dd419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4112050 con 0x7f7dd4103760 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4111ed0 con 0x7f7dd4102db0 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd9ce1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41121d0 con 0x7f7dd41082e0 2024-09-19T09:35:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34736/0 (socket says 172.21.15.28:34736) 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd2ffd640 1 -- 172.21.15.28:0/1001123419 learned_addr learned my addr 172.21.15.28:0/1001123419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 0x7f7dd41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.899+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 0x7f7dd419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 -- 172.21.15.28:0/1001123419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 msgr2=0x7f7dd419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 0x7f7dd419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 -- 172.21.15.28:0/1001123419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 msgr2=0x7f7dd41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 0x7f7dd41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 -- 172.21.15.28:0/1001123419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd419d560 con 0x7f7dd4103760 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 0x7f7dd41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 0x7f7dd419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005940 tx=0x7f7dc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd0ff9640 1 -- 172.21.15.28:0/1001123419 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc803f070 con 0x7f7dd4103760 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd419d7f0 con 0x7f7dd4103760 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd0ff9640 1 -- 172.21.15.28:0/1001123419 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc8031c30 con 0x7f7dd4103760 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd0ff9640 1 -- 172.21.15.28:0/1001123419 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8035030 con 0x7f7dd4103760 2024-09-19T09:35:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.900+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd4100940 con 0x7f7dd4103760 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7dd0ff9640 1 -- 172.21.15.28:0/1001123419 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7dc8042070 con 0x7f7dd4103760 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 Processor -- start 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 -- start start 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdebbb40 0x55f6cdeba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdec7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f6cde202f0 con 0x55f6cdec5370 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f6cddc7cf0 con 0x55f6cdebbb40 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6cde1c2b0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.903+0000 7f7dd0ff9640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 0x7f7dbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53954/0 (socket says 172.21.15.28:53954) 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 -- 172.21.15.28:0/675678291 learned_addr learned my addr 172.21.15.28:0/675678291 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdec7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 0x7f7dbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdebbb40 0x55f6cdeba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd0ff9640 1 -- 172.21.15.28:0/1001123419 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7dc80f0000 con 0x7f7dd4103760 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 -- 172.21.15.28:0/675678291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdebbb40 msgr2=0x55f6cdeba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdebbb40 0x55f6cdeba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 -- 172.21.15.28:0/675678291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 msgr2=0x55f6cdec7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdec7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 -- 172.21.15.28:0/675678291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cddac3a0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 0x7f7dbc0a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005fd0 tx=0x7f7dcc009750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.904+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40115f0 tx=0x7f7dc4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7177fe84b3ef1c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.905+0000 7f7db9ffb640 1 -- 172.21.15.28:0/675678291 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401e070 con 0x55f6cdeba5d0 2024-09-19T09:35:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.905+0000 7f7db9ffb640 1 -- 172.21.15.28:0/675678291 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc400fb20 con 0x55f6cdeba5d0 2024-09-19T09:35:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.905+0000 7f7db9ffb640 1 -- 172.21.15.28:0/675678291 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4019c10 con 0x55f6cdeba5d0 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.905+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/675678291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 msgr2=0x55f6cdebaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.905+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40115f0 tx=0x7f7dc4011ab0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/675678291 shutdown_connections 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdec7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdebaa10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/675678291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdebbb40 0x55f6cdeba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/675678291 >> 172.21.15.28:0/675678291 conn(0x55f6cde83310 msgr2=0x55f6cdeb6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/675678291 shutdown_connections 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/675678291 wait complete. 2024-09-19T09:35:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 -- start start 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.906+0000 7f7ddc0ba740 1 Processor -- start 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdec4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdec2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f6cdec8070 con 0x55f6cdebbb40 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f6cdddb060 con 0x55f6cdeba5d0 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6cdec81f0 con 0x55f6cdec5370 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53956/0 (socket says 172.21.15.28:53956) 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 -- 172.21.15.28:0/2132919016 learned_addr learned my addr 172.21.15.28:0/2132919016 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdec4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdec2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7ddc0ba740 10 client.0 _ll_register_callbacks cb 0x55f6cdb2e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 -- 172.21.15.28:0/2132919016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 msgr2=0x55f6cdec4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdec4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 -- 172.21.15.28:0/2132919016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 msgr2=0x55f6cdec2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdec2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.907+0000 7f7dd3fff640 1 -- 172.21.15.28:0/2132919016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cdb913c0 con 0x55f6cdec5370 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009a10 tx=0x7f7dcc00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49eaddd18d67a57a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc01be50 con 0x55f6cdec5370 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7dcc00e3e0 con 0x55f6cdec5370 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc400fb20 con 0x55f6cdec5370 2024-09-19T09:35:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7dcc00e5e0 con 0x55f6cdec5370 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7ddc0ba740 10 client.25882 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f6cded3af0 con 0x55f6cdec5370 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.908+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc400fb20 con 0x55f6cdec5370 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f7dcc028b10 con 0x55f6cdec5370 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 10 client.25882 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 10 client.25882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 20 client.25882 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cded4f00 0x55f6cded7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 4 client.25882 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cde2d860 con 0x55f6cded4f00 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cded4f00 0x55f6cded7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cded4f00 0x55f6cded7330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ed327ca7666814e server_cookie=628abbbc7a2d6750 in_seq=0 out_seq=0 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.909+0000 7f7db2ffd640 10 client.25882 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7db2ffd640 1 -- 172.21.15.28:0/2132919016 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6cde2d860 con 0x55f6cded4f00 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7db2ffd640 10 client.25882 handle_command_reply: tid=0 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 client.25882 shutdown 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 msgr2=0x55f6cdec2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009a10 tx=0x7f7dcc00f7f0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 shutdown_connections 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cded4f00 0x55f6cded7330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdec5370 0x55f6cdec2600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdec2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2132919016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdec4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7db2ffd640 0 client.25882 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:35:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 >> 172.21.15.28:0/2132919016 conn(0x55f6cde83310 msgr2=0x55f6cdebc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 shutdown_connections 2024-09-19T09:35:45.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.910+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2132919016 wait complete. 2024-09-19T09:35:45.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.911+0000 7f7ddc0ba740 20 client.25882 trim_cache size 0 max 16384 2024-09-19T09:35:45.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.924+0000 7f7ddc0ba740 1 Processor -- start 2024-09-19T09:35:45.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.924+0000 7f7ddc0ba740 1 -- start start 2024-09-19T09:35:45.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdeba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41ab300 con 0x55f6cdec5370 2024-09-19T09:35:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41acf00 con 0x55f6cdeba5d0 2024-09-19T09:35:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41ab9d0 con 0x55f6cdebbb40 2024-09-19T09:35:45.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53958/0 (socket says 172.21.15.28:53958) 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34762/0 (socket says 172.21.15.28:34762) 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd2ffd640 1 -- 172.21.15.28:0/2833521005 learned_addr learned my addr 172.21.15.28:0/2833521005 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdeba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 -- 172.21.15.28:0/2833521005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 msgr2=0x55f6cded1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 -- 172.21.15.28:0/2833521005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 msgr2=0x55f6cdeb6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd37fe640 1 -- 172.21.15.28:0/2833521005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cdec3390 con 0x55f6cdeba5d0 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.925+0000 7f7dd3fff640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.926+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdeba9b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc401c040 tx=0x7f7dc4019890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5bcbdb9fdda41c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.926+0000 7f7db97fa640 1 -- 172.21.15.28:0/2833521005 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401e040 con 0x55f6cdeba5d0 2024-09-19T09:35:45.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.926+0000 7f7db97fa640 1 -- 172.21.15.28:0/2833521005 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc400fb20 con 0x55f6cdeba5d0 2024-09-19T09:35:45.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.926+0000 7f7db97fa640 1 -- 172.21.15.28:0/2833521005 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40050c0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2833521005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 msgr2=0x55f6cdeba9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdeba9b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc401c040 tx=0x7f7dc4019890 comp rx=0 tx=0).stop 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2833521005 shutdown_connections 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdec5370 0x55f6cdeb6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdebbb40 0x55f6cded1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/2833521005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdeba5d0 0x55f6cdeba9b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2833521005 >> 172.21.15.28:0/2833521005 conn(0x55f6cde83310 msgr2=0x55f6cdeb1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2833521005 shutdown_connections 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.927+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/2833521005 wait complete. 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 -- start start 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 Processor -- start 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdecdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdec5370 0x55f6cdebea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41ac770 con 0x55f6cdebbb40 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41ac0a0 con 0x55f6cdec5370 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41a92e0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7ddc0ba740 10 client.0 _ll_register_callbacks cb 0x7f7dcc009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53964/0 (socket says 172.21.15.28:53964) 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.928+0000 7f7dd37fe640 1 -- 172.21.15.28:0/849947021 learned_addr learned my addr 172.21.15.28:0/849947021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdecdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 -- 172.21.15.28:0/849947021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdec5370 msgr2=0x55f6cdebea50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdec5370 0x55f6cdebea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 -- 172.21.15.28:0/849947021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 msgr2=0x55f6cdecdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdecdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 -- 172.21.15.28:0/849947021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cdebf280 con 0x55f6cdeba5d0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7dd37fe640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400fef0 tx=0x7f7dc400ff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50289a8ef2d53328 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401e070 con 0x55f6cdeba5d0 2024-09-19T09:35:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7dc4009cc0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.929+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc400fb20 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7dc4019900 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 10 client.25888 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f6cdf3f7f0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc4019aa0 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f7dc402b030 con 0x55f6cdeba5d0 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 10 client.25888 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 10 client.25888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-19T09:35:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 20 client.25888 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cdf3fcb0 0x55f6cdf420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 4 client.25888 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 --> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7dc400fb20 con 0x55f6cdf3fcb0 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.930+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cdf3fcb0 0x55f6cdf420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7db2ffd640 10 client.25888 ms_handle_connect on v2:172.21.15.78:6834/2086564778 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7dd2ffd640 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cdf3fcb0 0x55f6cdf420a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab8f260f57ddac26 server_cookie=9dbbed1bc2aa4b6f in_seq=0 out_seq=0 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7db2ffd640 1 -- 172.21.15.28:0/849947021 <== mds.3 v2:172.21.15.78:6834/2086564778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7758 (crc 0 0 0) 0x7f7dc400fb20 con 0x55f6cdf3fcb0 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7db2ffd640 10 client.25888 handle_command_reply: tid=0 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 client.25888 shutdown 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 msgr2=0x55f6cdecd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400fef0 tx=0x7f7dc400ff20 comp rx=0 tx=0).stop 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 shutdown_connections 2024-09-19T09:35:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:6834/2086564778,v1:172.21.15.78:6835/2086564778] conn(0x55f6cdf3fcb0 0x55f6cdf420a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6cdec5370 0x55f6cdebea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6cdebbb40 0x55f6cdecdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7db2ffd640 0 client.25888 ms_handle_reset on v2:172.21.15.78:6834/2086564778 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 --2- 172.21.15.28:0/849947021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6cdeba5d0 0x55f6cdecd760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.931+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 >> 172.21.15.28:0/849947021 conn(0x55f6cde83310 msgr2=0x55f6cdeb54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 shutdown_connections 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7ddc0ba740 1 -- 172.21.15.28:0/849947021 wait complete. 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7ddc0ba740 20 client.25888 trim_cache size 0 max 16384 2024-09-19T09:35:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 64214544, 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635501, 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml" 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 78899554, 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635469, 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml" 2024-09-19T09:35:45.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 141973061, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635513, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml" 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 146550252, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635497, 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml" 2024-09-19T09:35:45.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 189050940, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635489, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml" 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 292116881, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635521, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml" 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 293957799, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635461, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml" 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 746072299, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635427, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml" 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.943 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 786826450, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635442, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml" 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 810526816, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635432, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml" 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 837056959, 2024-09-19T09:35:45.944 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635422, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml" 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1154397641, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635517, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml" 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1184106603, 2024-09-19T09:35:45.945 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635452, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml" 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1185431172, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635509, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml" 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1322583734, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635537, 2024-09-19T09:35:45.946 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml" 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1509241879, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635485, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml" 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1610455798, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635505, 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml" 2024-09-19T09:35:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1708148825, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635412, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml" 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1872315549, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635525, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml" 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2168583767, 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635407, 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml" 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2304095153, 2024-09-19T09:35:45.949 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635477, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml" 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2476168433, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635541, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml" 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2611167917, 2024-09-19T09:35:45.950 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635457, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml" 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2923359205, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635437, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml" 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2987978973, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635465, 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml" 2024-09-19T09:35:45.951 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3204859604, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635533, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml" 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3490786680, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635473, 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml" 2024-09-19T09:35:45.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3734261190, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635529, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml" 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3907198819, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635493, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml" 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.953 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4042894018, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635447, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml" 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4102236919, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635481, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml" 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2024-09-19T09:35:45.954 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4187979856, 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511635417, 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml" 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 msgr2=0x7f7dbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 0x7f7dbc0a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005fd0 tx=0x7f7dcc009750 comp rx=0 tx=0).stop 2024-09-19T09:35:45.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 msgr2=0x7f7dd4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.932+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005940 tx=0x7f7dc8003c10 comp rx=0 tx=0).stop 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 shutdown_connections 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7dbc0a3240 0x7f7dbc0a5700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd41082e0 0x7f7dd419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7dd4103760 0x7f7dd4198af0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 --2- 172.21.15.28:0/1001123419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7dd4102db0 0x7f7dd41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 >> 172.21.15.28:0/1001123419 conn(0x7f7dd40fc7d0 msgr2=0x7f7dd4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 shutdown_connections 2024-09-19T09:35:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:45.933+0000 7f7dd9ce1640 1 -- 172.21.15.28:0/1001123419 wait complete. 2024-09-19T09:35:47.270 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:35:47.655 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:47 smithi028 ceph-mon[30972]: pgmap v431: 97 pgs: 97 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 1.33k op/s 2024-09-19T09:35:47.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:47 smithi038 ceph-mon[35271]: pgmap v431: 97 pgs: 97 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 1.33k op/s 2024-09-19T09:35:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:47 smithi078 ceph-mon[35619]: pgmap v431: 97 pgs: 97 active+clean; 217 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 1.33k op/s 2024-09-19T09:35:48.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:48 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:35:48.747 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:48 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:35:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:48 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:35:49.039 INFO:tasks.workunit.client.0.smithi028.stdout: 77 1602262 699 1059631 438 754936 1708 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 -- 172.21.15.28:0/436783050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8101430 msgr2=0x7ff5f806a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 --2- 172.21.15.28:0/436783050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8101430 0x7ff5f806a790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009ca0 tx=0x7ff5f002a8d0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 -- 172.21.15.28:0/436783050 shutdown_connections 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 --2- 172.21.15.28:0/436783050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8101430 0x7ff5f806a790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 --2- 172.21.15.28:0/436783050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 --2- 172.21.15.28:0/436783050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8106a90 0x7ff5f8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.440+0000 7ff5fce62640 1 -- 172.21.15.28:0/436783050 >> 172.21.15.28:0/436783050 conn(0x7ff5f80fc7d0 msgr2=0x7ff5f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.441+0000 7ff5fce62640 1 -- 172.21.15.28:0/436783050 shutdown_connections 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.442+0000 7ff5fce62640 1 -- 172.21.15.28:0/436783050 wait complete. 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.443+0000 7ff5fce62640 1 Processor -- start 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.443+0000 7ff5fce62640 1 -- start start 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8106a90 0x7ff5f81091a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 0x7ff5f80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810fe40 con 0x7ff5f8072c60 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810fcc0 con 0x7ff5f8106a90 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.444+0000 7ff5fce62640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810ffc0 con 0x7ff5f8100a90 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.446+0000 7ff5f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.446+0000 7ff5f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53966/0 (socket says 172.21.15.28:53966) 2024-09-19T09:35:49.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.446+0000 7ff5f659c640 1 -- 172.21.15.28:0/3010149562 learned_addr learned my addr 172.21.15.28:0/3010149562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.446+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 0x7ff5f80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f659c640 1 -- 172.21.15.28:0/3010149562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8106a90 msgr2=0x7ff5f81091a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f659c640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8106a90 0x7ff5f81091a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f659c640 1 -- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 msgr2=0x7ff5f80730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f659c640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 0x7ff5f80730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f659c640 1 -- 172.21.15.28:0/3010149562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8073620 con 0x7ff5f8100a90 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.447+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 0x7ff5f80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:49.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.449+0000 7ff5f659c640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d770 tx=0x7ff5e800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.453+0000 7ff5e77fe640 1 -- 172.21.15.28:0/3010149562 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8017070 con 0x7ff5f8100a90 2024-09-19T09:35:49.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.453+0000 7ff5e77fe640 1 -- 172.21.15.28:0/3010149562 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8005500 con 0x7ff5f8100a90 2024-09-19T09:35:49.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.453+0000 7ff5e77fe640 1 -- 172.21.15.28:0/3010149562 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004d00 con 0x7ff5f8100a90 2024-09-19T09:35:49.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.456+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f8073910 con 0x7ff5f8100a90 2024-09-19T09:35:49.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.456+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f8073e00 con 0x7ff5f8100a90 2024-09-19T09:35:49.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.464+0000 7ff5ff23b740 1 Processor -- start 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 -- start start 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41ace6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41acf1370 0x55c41acf3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c41ac4c2f0 con 0x55c41ace65d0 2024-09-19T09:35:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c41abf3cf0 con 0x55c41acf1370 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5ff23b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c41ac482b0 con 0x55c41ace7b40 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.466+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.466+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53968/0 (socket says 172.21.15.28:53968) 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.466+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/1501176178 learned_addr learned my addr 172.21.15.28:0/1501176178 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.465+0000 7ff5e77fe640 1 -- 172.21.15.28:0/3010149562 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff5e8009bd0 con 0x7ff5f8100a90 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/1501176178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41acf1370 msgr2=0x55c41acf3760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41acf1370 0x55c41acf3760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/1501176178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 msgr2=0x55c41ace6a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41ace6a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/1501176178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41abd83a0 con 0x55c41ace7b40 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0035000 tx=0x7ff5f0003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab237bdd20a6a19e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.467+0000 7ff5e4ff9640 1 -- 172.21.15.28:0/1501176178 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0004040 con 0x55c41ace7b40 2024-09-19T09:35:49.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.468+0000 7ff5e77fe640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 0x7ff5e00a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.469+0000 7ff5e77fe640 1 -- 172.21.15.28:0/3010149562 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff5e80cf560 con 0x7ff5f8100a90 2024-09-19T09:35:49.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.469+0000 7ff5f659c640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 0x7ff5e00a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.469+0000 7ff5e4ff9640 1 -- 172.21.15.28:0/1501176178 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0036040 con 0x55c41ace7b40 2024-09-19T09:35:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.470+0000 7ff5ff23b740 1 -- 172.21.15.28:0/1501176178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 msgr2=0x55c41ace6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.470+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0035000 tx=0x7ff5f0003e40 comp rx=0 tx=0).stop 2024-09-19T09:35:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5ff23b740 1 -- 172.21.15.28:0/1501176178 shutdown_connections 2024-09-19T09:35:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41acf1370 0x55c41acf3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41ace6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/1501176178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41ace6090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5ff23b740 1 -- 172.21.15.28:0/1501176178 >> 172.21.15.28:0/1501176178 conn(0x55c41acaf310 msgr2=0x55c41ace2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.471+0000 7ff5f659c640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 0x7ff5e00a5910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80985a0 tx=0x7ff5e8097f20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.474+0000 7ff5ff23b740 1 -- 172.21.15.28:0/1501176178 shutdown_connections 2024-09-19T09:35:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.478+0000 7ff5ff23b740 1 -- 172.21.15.28:0/1501176178 wait complete. 2024-09-19T09:35:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.478+0000 7ff5ff23b740 1 -- start start 2024-09-19T09:35:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.479+0000 7ff5ff23b740 1 Processor -- start 2024-09-19T09:35:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.479+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.479+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.479+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 0x55c41acee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 0x55c41acee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 0x55c41acee3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34802/0 (socket says 172.21.15.28:34802) 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53978/0 (socket says 172.21.15.28:53978) 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/248177435 learned_addr learned my addr 172.21.15.28:0/248177435 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c41acf3f80 con 0x55c41acf1370 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c41ac07060 con 0x55c41ace65d0 2024-09-19T09:35:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c41acf4100 con 0x55c41ace7b40 2024-09-19T09:35:49.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.480+0000 7ff5ff23b740 10 client.0 _ll_register_callbacks cb 0x55c41a95a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:49.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.482+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/248177435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 msgr2=0x55c41acf0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:49.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/248177435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 msgr2=0x55c41acee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:49.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 0x55c41acee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41a9bd3c0 con 0x55c41ace7b40 2024-09-19T09:35:49.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.483+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5f810a250 tx=0x7ff5ec009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1163c67715baf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.487+0000 7ff5ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:49.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.487+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec011420 con 0x55c41ace7b40 2024-09-19T09:35:49.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.487+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f0036040 con 0x55c41ace7b40 2024-09-19T09:35:49.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.487+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff5ec0115c0 con 0x55c41ace7b40 2024-09-19T09:35:49.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.491+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5ec011800 con 0x55c41ace7b40 2024-09-19T09:35:49.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.493+0000 7ff5ff23b740 10 client.25897 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:49.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.493+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c41acff9f0 con 0x55c41ace7b40 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0036040 con 0x55c41ace7b40 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7ff5ec010bf0 con 0x55c41ace7b40 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 10 client.25897 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 10 client.25897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 20 client.25897 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad01370 0x55c41ad037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 4 client.25897 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.497+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c41ac59860 con 0x55c41ad01370 2024-09-19T09:35:49.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.500+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad01370 0x55c41ad037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.500+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad01370 0x55c41ad037b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93f54f0a4c5d9bd9 server_cookie=ab0447071f97d9a2 in_seq=0 out_seq=0 2024-09-19T09:35:49.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.500+0000 7ff5ddffb640 10 client.25897 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:35:49.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.501+0000 7ff5ddffb640 1 -- 172.21.15.28:0/248177435 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c41ac59860 con 0x55c41ad01370 2024-09-19T09:35:49.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.501+0000 7ff5ddffb640 10 client.25897 handle_command_reply: tid=0 2024-09-19T09:35:49.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.501+0000 7ff5ff23b740 1 client.25897 shutdown 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 msgr2=0x55c41acedeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5f810a250 tx=0x7ff5ec009490 comp rx=0 tx=0).stop 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 shutdown_connections 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad01370 0x55c41ad037b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41acf1370 0x55c41acee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41ace7b40 0x55c41acedeb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/248177435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 >> 172.21.15.28:0/248177435 conn(0x55c41acaf310 msgr2=0x55c41ace8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.503+0000 7ff5ddffb640 0 client.25897 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:35:49.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.506+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 shutdown_connections 2024-09-19T09:35:49.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.510+0000 7ff5ff23b740 1 -- 172.21.15.28:0/248177435 wait complete. 2024-09-19T09:35:49.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.511+0000 7ff5ff23b740 20 client.25897 trim_cache size 0 max 16384 2024-09-19T09:35:49.526 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:49 smithi028 ceph-mon[30972]: pgmap v432: 97 pgs: 97 active+clean; 216 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.26k op/s 2024-09-19T09:35:49.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.541+0000 7ff5ff23b740 1 Processor -- start 2024-09-19T09:35:49.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.543+0000 7ff5ff23b740 1 -- start start 2024-09-19T09:35:49.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.543+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41acfd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace7b40 0x55c41ad04050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41ad04590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5ff23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f811c080 con 0x55c41ace65d0 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41ad04590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41ad04590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53988/0 (socket says 172.21.15.28:53988) 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/335517282 learned_addr learned my addr 172.21.15.28:0/335517282 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81a49e0 con 0x55c41ace7b40 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.544+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81a34b0 con 0x55c41acf1370 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41acfd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/335517282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 msgr2=0x55c41ad04590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41ad04590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/335517282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace7b40 msgr2=0x55c41ad04050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:35:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace7b40 0x55c41ad04050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.545+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/335517282 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41acec400 con 0x55c41ace65d0 2024-09-19T09:35:49.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.546+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41acfd290 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009bf0 tx=0x7ff5f0030830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb144981f5bb5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.551+0000 7ff5dffff640 1 -- 172.21.15.28:0/335517282 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f004a040 con 0x55c41ace65d0 2024-09-19T09:35:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.551+0000 7ff5dffff640 1 -- 172.21.15.28:0/335517282 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0036040 con 0x55c41ace65d0 2024-09-19T09:35:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.551+0000 7ff5dffff640 1 -- 172.21.15.28:0/335517282 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f002f580 con 0x55c41ace65d0 2024-09-19T09:35:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.555+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 msgr2=0x55c41acfd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.555+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41acfd290 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009bf0 tx=0x7ff5f0030830 comp rx=0 tx=0).stop 2024-09-19T09:35:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.556+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 shutdown_connections 2024-09-19T09:35:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.556+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace7b40 0x55c41ad04050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.556+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41ad04590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.556+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/335517282 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace65d0 0x55c41acfd290 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.556+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 >> 172.21.15.28:0/335517282 conn(0x55c41acaf310 msgr2=0x55c41ace5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.563+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 shutdown_connections 2024-09-19T09:35:49.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.568+0000 7ff5ff23b740 1 -- 172.21.15.28:0/335517282 wait complete. 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.568+0000 7ff5ff23b740 1 -- start start 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.569+0000 7ff5ff23b740 1 Processor -- start 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.569+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5ff23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:40840/0 (socket says 172.21.15.28:40840) 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5f5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5f5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34826/0 (socket says 172.21.15.28:34826) 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.570+0000 7ff5f6d9d640 1 -- 172.21.15.28:0/358981804 learned_addr learned my addr 172.21.15.28:0/358981804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.571+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41acf4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.571+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81a4250 con 0x55c41ace7b40 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.571+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81a3b80 con 0x55c41ace65d0 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.571+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f811a060 con 0x55c41acf1370 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 msgr2=0x55c41acf4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41acf4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/358981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 msgr2=0x55c41acf9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41acf5130 con 0x55c41ace7b40 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.572+0000 7ff5f5d9b640 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55c41acf09f0 tx=0x7ff5ec019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4bf82499f21d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.573+0000 7ff5ff23b740 10 client.0 _ll_register_callbacks cb 0x7ff5ec006df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:49.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.573+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f0036040 con 0x55c41ace7b40 2024-09-19T09:35:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.576+0000 7ff5ddffb640 10 client.16350 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.576+0000 7ff5ddffb640 1 -- 172.21.15.28:0/358981804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec006b10 con 0x55c41ace7b40 2024-09-19T09:35:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.576+0000 7ff5ddffb640 1 -- 172.21.15.28:0/358981804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7ff5ec0199e0 con 0x55c41ace7b40 2024-09-19T09:35:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.576+0000 7ff5ddffb640 1 -- 172.21.15.28:0/358981804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5ec02a070 con 0x55c41ace7b40 2024-09-19T09:35:49.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.579+0000 7ff5ff23b740 10 client.16350 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:49.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.579+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55c41ad8da60 con 0x55c41ace7b40 2024-09-19T09:35:49.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.580+0000 7ff5ddffb640 1 -- 172.21.15.28:0/358981804 <== mon.0 v2:172.21.15.28:3300/0 4 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7ff5ec019c40 con 0x55c41ace7b40 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 10 client.16350 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 20 client.16350 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad8e0c0 0x55c41ad90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 4 client.16350 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:35:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.582+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 --> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff5f0036040 con 0x55c41ad8e0c0 2024-09-19T09:35:49.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.585+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad8e0c0 0x55c41ad90550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:49.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.593+0000 7ff5f6d9d640 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad8e0c0 0x55c41ad90550 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d03643f37478a4f3 server_cookie=807138a510b91dfd in_seq=0 out_seq=0 2024-09-19T09:35:49.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.593+0000 7ff5ddffb640 10 client.16350 ms_handle_connect on v2:172.21.15.78:6836/2648819818 2024-09-19T09:35:49.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.595+0000 7ff5ddffb640 1 -- 172.21.15.28:0/358981804 <== mds.2 v2:172.21.15.78:6836/2648819818 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff5f0036040 con 0x55c41ad8e0c0 2024-09-19T09:35:49.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.596+0000 7ff5ddffb640 10 client.16350 handle_command_reply: tid=0 2024-09-19T09:35:49.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.597+0000 7ff5ff23b740 1 client.16350 shutdown 2024-09-19T09:35:49.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.601+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 msgr2=0x55c41acf43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.601+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55c41acf09f0 tx=0x7ff5ec019770 comp rx=0 tx=0).stop 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 shutdown_connections 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:6836/2648819818,v1:172.21.15.78:6837/2648819818] conn(0x55c41ad8e0c0 0x55c41ad90550 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c41acf1370 0x55c41acf4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c41ace7b40 0x55c41acf43c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 --2- 172.21.15.28:0/358981804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c41ace65d0 0x55c41acf9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ddffb640 0 client.16350 ms_handle_reset on v2:172.21.15.78:6836/2648819818 2024-09-19T09:35:49.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.602+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 >> 172.21.15.28:0/358981804 conn(0x55c41acaf310 msgr2=0x55c41ad03e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.603+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 shutdown_connections 2024-09-19T09:35:49.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.603+0000 7ff5ff23b740 1 -- 172.21.15.28:0/358981804 wait complete. 2024-09-19T09:35:49.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.603+0000 7ff5ff23b740 20 client.16350 trim_cache size 0 max 16384 2024-09-19T09:35:49.607 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:35:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 msgr2=0x7ff5e00a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 0x7ff5e00a5910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80985a0 tx=0x7ff5e8097f20 comp rx=0 tx=0).stop 2024-09-19T09:35:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 msgr2=0x7ff5f8108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d770 tx=0x7ff5e800dc30 comp rx=0 tx=0).stop 2024-09-19T09:35:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 shutdown_connections 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5e00a3450 0x7ff5e00a5910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f8106a90 0x7ff5f81091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5f8072c60 0x7ff5f80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 --2- 172.21.15.28:0/3010149562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5f8100a90 0x7ff5f8108c60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 >> 172.21.15.28:0/3010149562 conn(0x7ff5f80fc7d0 msgr2=0x7ff5f810f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 shutdown_connections 2024-09-19T09:35:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:49.609+0000 7ff5fce62640 1 -- 172.21.15.28:0/3010149562 wait complete. 2024-09-19T09:35:49.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:49 smithi038 ceph-mon[35271]: pgmap v432: 97 pgs: 97 active+clean; 216 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.26k op/s 2024-09-19T09:35:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:49 smithi078 ceph-mon[35619]: pgmap v432: 97 pgs: 97 active+clean; 216 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.26k op/s 2024-09-19T09:35:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:50 smithi028 ceph-mon[30972]: pgmap v433: 97 pgs: 97 active+clean; 218 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.22k op/s 2024-09-19T09:35:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:35:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:35:50.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:35:50.627 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:35:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:50 smithi038 ceph-mon[35271]: pgmap v433: 97 pgs: 97 active+clean; 218 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.22k op/s 2024-09-19T09:35:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:35:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:35:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:35:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:50 smithi078 ceph-mon[35619]: pgmap v433: 97 pgs: 97 active+clean; 218 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.22k op/s 2024-09-19T09:35:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:35:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:35:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:35:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 -- 172.21.15.28:0/387205790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 msgr2=0x7fcd50108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 --2- 172.21.15.28:0/387205790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd50108e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009ca0 tx=0x7fcd4802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 -- 172.21.15.28:0/387205790 shutdown_connections 2024-09-19T09:35:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 --2- 172.21.15.28:0/387205790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 0x7fcd501020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 --2- 172.21.15.28:0/387205790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd50108e70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 --2- 172.21.15.28:0/387205790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 0x7fcd50100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 -- 172.21.15.28:0/387205790 >> 172.21.15.28:0/387205790 conn(0x7fcd500fc190 msgr2=0x7fcd500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 -- 172.21.15.28:0/387205790 shutdown_connections 2024-09-19T09:35:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.411+0000 7fcd583e8640 1 -- 172.21.15.28:0/387205790 wait complete. 2024-09-19T09:35:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 Processor -- start 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 -- start start 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 0x7fcd50196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 0x7fcd50197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5010c700 con 0x7fcd50100a30 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5010c580 con 0x7fcd50101c20 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd583e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5010c880 con 0x7fcd501025e0 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd56985640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd56184640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 0x7fcd50196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd56985640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54028/0 (socket says 172.21.15.28:54028) 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd56985640 1 -- 172.21.15.28:0/262685471 learned_addr learned my addr 172.21.15.28:0/262685471 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.412+0000 7fcd55983640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 0x7fcd50197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 -- 172.21.15.28:0/262685471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 msgr2=0x7fcd50197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 0x7fcd50197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 -- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 msgr2=0x7fcd50196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 0x7fcd50196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 -- 172.21.15.28:0/262685471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5019e950 con 0x7fcd501025e0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd56985640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009430 tx=0x7fcd480319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd477fe640 1 -- 172.21.15.28:0/262685471 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4804c070 con 0x7fcd501025e0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd477fe640 1 -- 172.21.15.28:0/262685471 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd48003d80 con 0x7fcd501025e0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.413+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5006a230 con 0x7fcd501025e0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.414+0000 7fcd477fe640 1 -- 172.21.15.28:0/262685471 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004860 con 0x7fcd501025e0 2024-09-19T09:35:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.414+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5006a710 con 0x7fcd501025e0 2024-09-19T09:35:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.417+0000 7fcd477fe640 1 -- 172.21.15.28:0/262685471 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcd4804f070 con 0x7fcd501025e0 2024-09-19T09:35:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.417+0000 7fcd5b262740 1 Processor -- start 2024-09-19T09:35:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 -- start start 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde0031d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0037360 0x55fde0039820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55fddff952f0 con 0x55fde0031990 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fddff3d270 con 0x55fde00322d0 2024-09-19T09:35:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd5b262740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fddff912b0 con 0x55fde0037360 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:40864/0 (socket says 172.21.15.28:40864) 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 -- 172.21.15.28:0/1799819839 learned_addr learned my addr 172.21.15.28:0/1799819839 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd55983640 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde0031d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 -- 172.21.15.28:0/1799819839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0037360 msgr2=0x55fde0039820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0037360 0x55fde0039820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 -- 172.21.15.28:0/1799819839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 msgr2=0x55fde0031d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde0031d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd56184640 1 -- 172.21.15.28:0/1799819839 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fddff3c340 con 0x55fde00322d0 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.418+0000 7fcd55983640 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde0031d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.419+0000 7fcd56184640 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00b750 tx=0x7fcd4c00bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5334a066fb4c6eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.419+0000 7fcd44bfe640 1 -- 172.21.15.28:0/1799819839 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00be60 con 0x55fde00322d0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.419+0000 7fcd44bfe640 1 -- 172.21.15.28:0/1799819839 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4c015070 con 0x55fde00322d0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.419+0000 7fcd44bfe640 1 -- 172.21.15.28:0/1799819839 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0116a0 con 0x55fde00322d0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.419+0000 7fcd477fe640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 0x7fcd3c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd55983640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 0x7fcd3c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd477fe640 1 -- 172.21.15.28:0/262685471 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcd480f48b0 con 0x7fcd501025e0 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 -- 172.21.15.28:0/1799819839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 msgr2=0x55fde0036d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00b750 tx=0x7fcd4c00bc10 comp rx=0 tx=0).stop 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 -- 172.21.15.28:0/1799819839 shutdown_connections 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0036d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0037360 0x55fde0039820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1799819839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde0031d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd5b262740 1 -- 172.21.15.28:0/1799819839 >> 172.21.15.28:0/1799819839 conn(0x55fde002d000 msgr2=0x55fde002d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.420+0000 7fcd55983640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 0x7fcd3c0a5800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009b50 tx=0x7fcd40010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- 172.21.15.28:0/1799819839 shutdown_connections 2024-09-19T09:35:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- 172.21.15.28:0/1799819839 wait complete. 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- start start 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 Processor -- start 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0044360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde00448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55fde003a330 con 0x55fde0037360 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fddff1e290 con 0x55fde00322d0 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fde003a4b0 con 0x55fde0031990 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.421+0000 7fcd5b262740 10 client.0 _ll_register_callbacks cb 0x55fddff4f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34850/0 (socket says 172.21.15.28:34850) 2024-09-19T09:35:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 -- 172.21.15.28:0/14018767 learned_addr learned my addr 172.21.15.28:0/14018767 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd55983640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde00448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 -- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 msgr2=0x55fde0044360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56184640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0044360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0044360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 -- 172.21.15.28:0/14018767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 msgr2=0x55fde00448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde00448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 -- 172.21.15.28:0/14018767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fddfca8210 con 0x55fde0037360 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd55983640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde00448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56184640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0044360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd56985640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcd480c95e0 tx=0x7fcd480c0b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453df85dba02f126 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.422+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48075070 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcd48004b40 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4c015070 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd480c5250 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd5b262740 10 client.16362 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fddfd74620 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.423+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd48036030 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fcd48046030 con 0x55fde0037360 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 10 client.16362 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 20 client.16362 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde0050170 0x55fde00525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 4 client.16362 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd4c015070 con 0x55fde0050170 2024-09-19T09:35:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.424+0000 7fcd56184640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde0050170 0x55fde00525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.425+0000 7fcd56184640 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde0050170 0x55fde00525d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8158ab01c3258327 server_cookie=d7784695d4e696de in_seq=0 out_seq=0 2024-09-19T09:35:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.425+0000 7fcd39ffb640 10 client.16362 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd39ffb640 1 -- 172.21.15.28:0/14018767 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcd4c015070 con 0x55fde0050170 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd39ffb640 10 client.16362 handle_command_reply: tid=0 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 client.16362 shutdown 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 msgr2=0x55fde004b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcd480c95e0 tx=0x7fcd480c0b50 comp rx=0 tx=0).stop 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 shutdown_connections 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde00448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde0050170 0x55fde00525d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004b8e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 --2- 172.21.15.28:0/14018767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0044360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 >> 172.21.15.28:0/14018767 conn(0x55fde002d000 msgr2=0x55fde002e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd39ffb640 0 client.16362 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 shutdown_connections 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.427+0000 7fcd5b262740 1 -- 172.21.15.28:0/14018767 wait complete. 2024-09-19T09:35:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.428+0000 7fcd5b262740 20 client.16362 trim_cache size 0 max 16384 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.454+0000 7fcd5b262740 1 Processor -- start 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.454+0000 7fcd5b262740 1 -- start start 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.454+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde002e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.454+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 0x55fde0056e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd5b262740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a6ad0 con 0x55fde0031990 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd5b262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a86d0 con 0x55fde0037360 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54050/0 (socket says 172.21.15.28:54050) 2024-09-19T09:35:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 -- 172.21.15.28:0/2649452908 learned_addr learned my addr 172.21.15.28:0/2649452908 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a71a0 con 0x55fde00322d0 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd56184640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde002e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd56985640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 0x55fde0056e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 -- 172.21.15.28:0/2649452908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 msgr2=0x55fde0056e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 0x55fde0056e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 -- 172.21.15.28:0/2649452908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 msgr2=0x55fde002e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde002e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 -- 172.21.15.28:0/2649452908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde00327b0 con 0x55fde00322d0 2024-09-19T09:35:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd56985640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 0x55fde0056e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.455+0000 7fcd55983640 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd4001a5b0 tx=0x7fcd40019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3c4ddedb484f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.456+0000 7fcd3bfff640 1 -- 172.21.15.28:0/2649452908 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4002b070 con 0x55fde00322d0 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.456+0000 7fcd3bfff640 1 -- 172.21.15.28:0/2649452908 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd40004230 con 0x55fde00322d0 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.456+0000 7fcd3bfff640 1 -- 172.21.15.28:0/2649452908 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40019980 con 0x55fde00322d0 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 msgr2=0x55fde004b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd4001a5b0 tx=0x7fcd40019000 comp rx=0 tx=0).stop 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 shutdown_connections 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde0037360 0x55fde0056e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde00322d0 0x55fde004b8a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- 172.21.15.28:0/2649452908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0031990 0x55fde002e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 >> 172.21.15.28:0/2649452908 conn(0x55fde002d000 msgr2=0x55fde002df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 shutdown_connections 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- 172.21.15.28:0/2649452908 wait complete. 2024-09-19T09:35:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- start start 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 Processor -- start 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0049740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a7f40 con 0x55fde0037360 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a7870 con 0x55fde00322d0 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501a4ab0 con 0x55fde0031990 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.457+0000 7fcd5b262740 10 client.0 _ll_register_callbacks cb 0x7fcd48003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54058/0 (socket says 172.21.15.28:54058) 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56985640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 -- 172.21.15.28:0/1313042839 learned_addr learned my addr 172.21.15.28:0/1313042839 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd55983640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0049740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 -- 172.21.15.28:0/1313042839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 msgr2=0x55fde0049740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0049740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 -- 172.21.15.28:0/1313042839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 msgr2=0x55fde004a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 -- 172.21.15.28:0/1313042839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde004ad60 con 0x55fde0031990 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd56184640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00b750 tx=0x7fcd4c004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3949dc0f5d47f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c02e060 con 0x55fde0031990 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcd4c01a470 con 0x55fde0031990 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.458+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd40004230 con 0x55fde0031990 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c003e60 con 0x55fde0031990 2024-09-19T09:35:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 10 client.25912 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fde00bb170 con 0x55fde0031990 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd40004230 con 0x55fde0031990 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fcd4c0274f0 con 0x55fde0031990 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 10 client.25912 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 10 client.25912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 20 client.25912 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde00bba40 0x55fde00bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 4 client.25912 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fddffa2a20 con 0x55fde00bba40 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.459+0000 7fcd55983640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde00bba40 0x55fde00bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.460+0000 7fcd39ffb640 10 client.25912 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:51.460+0000 7fcd55983640 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde00bba40 0x55fde00bde60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4463773eae5b86 server_cookie=f000633d12e8aef3 in_seq=0 out_seq=0 2024-09-19T09:35:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.243+0000 7f0d052d7640 1 -- 172.21.15.28:0/2915790234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 msgr2=0x7f0d00100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.243+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2915790234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d00100ca0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001ab70 tx=0x7f0cf003b6d0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.245+0000 7f0d052d7640 1 -- 172.21.15.28:0/2915790234 shutdown_connections 2024-09-19T09:35:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.245+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2915790234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d001011e0 0x7f0d00105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.245+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2915790234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d00105ec0 0x7f0d00106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.245+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2915790234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d00100ca0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.245+0000 7f0d052d7640 1 -- 172.21.15.28:0/2915790234 >> 172.21.15.28:0/2915790234 conn(0x7f0d000fc7d0 msgr2=0x7f0d000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.246+0000 7f0d052d7640 1 -- 172.21.15.28:0/2915790234 shutdown_connections 2024-09-19T09:35:52.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.246+0000 7f0d052d7640 1 -- 172.21.15.28:0/2915790234 wait complete. 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.247+0000 7f0d052d7640 1 Processor -- start 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.247+0000 7f0d052d7640 1 -- start start 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.248+0000 7f0d052d7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d0010d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.248+0000 7f0d052d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.248+0000 7f0d052d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00105ec0 0x7f0d00072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00105ec0 0x7f0d00072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54070/0 (socket says 172.21.15.28:54070) 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 -- 172.21.15.28:0/2035568322 learned_addr learned my addr 172.21.15.28:0/2035568322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00109980 con 0x7f0d001008c0 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00109800 con 0x7f0d00105ec0 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00109b00 con 0x7f0d001011e0 2024-09-19T09:35:52.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 -- 172.21.15.28:0/2035568322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00105ec0 msgr2=0x7f0d00072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00105ec0 0x7f0d00072b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.249+0000 7f0cf7fff640 1 -- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 msgr2=0x7f0d0010d730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:35:52.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.250+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d0010d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.251+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d0010d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.251+0000 7f0cf7fff640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d00073170 con 0x7f0d001011e0 2024-09-19T09:35:52.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.251+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d0010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:52.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.258+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800b450 tx=0x7f0ce800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.262+0000 7f0cfcff9640 1 -- 172.21.15.28:0/2035568322 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8015020 con 0x7f0d001011e0 2024-09-19T09:35:52.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.262+0000 7f0cfcff9640 1 -- 172.21.15.28:0/2035568322 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8011d70 con 0x7f0d001011e0 2024-09-19T09:35:52.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.262+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d00073460 con 0x7f0d001011e0 2024-09-19T09:35:52.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.262+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0019aa10 con 0x7f0d001011e0 2024-09-19T09:35:52.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.266+0000 7f0cfcff9640 1 -- 172.21.15.28:0/2035568322 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800f340 con 0x7f0d001011e0 2024-09-19T09:35:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.268+0000 7f0d076b0740 1 Processor -- start 2024-09-19T09:35:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.270+0000 7f0cfcff9640 1 -- 172.21.15.28:0/2035568322 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0ce8018020 con 0x7f0d001011e0 2024-09-19T09:35:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 -- start start 2024-09-19T09:35:52.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58032060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d5803f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559d57f982f0 con 0x559d58033b10 2024-09-19T09:35:52.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559d57f3fcf0 con 0x559d580325a0 2024-09-19T09:35:52.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.271+0000 7f0d076b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d57f942b0 con 0x559d5803d340 2024-09-19T09:35:52.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.272+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.272+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:40900/0 (socket says 172.21.15.28:40900) 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.272+0000 7f0cfeffd640 1 -- 172.21.15.28:0/3125720989 learned_addr learned my addr 172.21.15.28:0/3125720989 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.275+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58032060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.279+0000 7f0cff7fe640 1 -- 172.21.15.28:0/3125720989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 msgr2=0x559d5803f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.283+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d5803f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d5803f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cff7fe640 1 -- 172.21.15.28:0/3125720989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 msgr2=0x559d580329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cff7fe640 1 -- 172.21.15.28:0/3125720989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d57f243a0 con 0x559d58033b10 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d5803f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cfcff9640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 0x7f0cec0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.284+0000 7f0cfcff9640 1 -- 172.21.15.28:0/2035568322 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0ce80cf360 con 0x7f0d001011e0 2024-09-19T09:35:52.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.285+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58032060 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d0010e360 tx=0x7f0cf8009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9022a941e6a41278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.285+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:52.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.285+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 0x7f0cec0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.289+0000 7f0cf57fa640 1 -- 172.21.15.28:0/3125720989 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800a8b0 con 0x559d58033b10 2024-09-19T09:35:52.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.289+0000 7f0cf57fa640 1 -- 172.21.15.28:0/3125720989 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8019070 con 0x559d58033b10 2024-09-19T09:35:52.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.289+0000 7f0cf57fa640 1 -- 172.21.15.28:0/3125720989 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8013380 con 0x559d58033b10 2024-09-19T09:35:52.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.293+0000 7f0d076b0740 1 -- 172.21.15.28:0/3125720989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 msgr2=0x559d58032060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.293+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58032060 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d0010e360 tx=0x7f0cf8009630 comp rx=0 tx=0).stop 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0d076b0740 1 -- 172.21.15.28:0/3125720989 shutdown_connections 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 0x7f0cec0a5910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001aac0 tx=0x7f0cf001a570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d580329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d5803f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3125720989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58032060 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.295+0000 7f0d076b0740 1 -- 172.21.15.28:0/3125720989 >> 172.21.15.28:0/3125720989 conn(0x559d57ffb310 msgr2=0x559d5802ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.299+0000 7f0d076b0740 1 -- 172.21.15.28:0/3125720989 shutdown_connections 2024-09-19T09:35:52.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.301+0000 7f0d076b0740 1 -- 172.21.15.28:0/3125720989 wait complete. 2024-09-19T09:35:52.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.301+0000 7f0d076b0740 1 -- start start 2024-09-19T09:35:52.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.301+0000 7f0d076b0740 1 Processor -- start 2024-09-19T09:35:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d58033b10 0x559d5803a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 0x559d5803a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559d58040040 con 0x559d5803d340 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559d57f53060 con 0x559d58033b10 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d580401c0 con 0x559d580325a0 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0d076b0740 10 client.0 _ll_register_callbacks cb 0x559d57ca68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54082/0 (socket says 172.21.15.28:54082) 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0cfeffd640 1 -- 172.21.15.28:0/3165202489 learned_addr learned my addr 172.21.15.28:0/3165202489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d58033b10 0x559d5803a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.306+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 0x559d5803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.307+0000 7f0cfeffd640 1 -- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d58033b10 msgr2=0x559d5803a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.307+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d58033b10 0x559d5803a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.307+0000 7f0cfeffd640 1 -- 172.21.15.28:0/3165202489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 msgr2=0x559d5803a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.307+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 0x559d5803a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.307+0000 7f0cfeffd640 1 -- 172.21.15.28:0/3165202489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d57d093c0 con 0x559d580325a0 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.309+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001ab70 tx=0x7f0cf003b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba3c73a8c4c4d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.309+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 0x559d5803a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:52.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.309+0000 7f0ce67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:52.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.311+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf005e020 con 0x559d580325a0 2024-09-19T09:35:52.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.311+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf8019070 con 0x559d580325a0 2024-09-19T09:35:52.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.311+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0cf0005150 con 0x559d580325a0 2024-09-19T09:35:52.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.311+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0005350 con 0x559d580325a0 2024-09-19T09:35:52.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.315+0000 7f0d076b0740 10 client.25918 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:52.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.315+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559d5804ba80 con 0x559d580325a0 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.320+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8019070 con 0x559d580325a0 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.321+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f0cf005a560 con 0x559d580325a0 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 10 client.25918 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 10 client.25918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 20 client.25918 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d5804cf40 0x559d5804f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 4 client.25918 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.323+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d57fa5860 con 0x559d5804cf40 2024-09-19T09:35:52.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.326+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d5804cf40 0x559d5804f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.339+0000 7f0cf7fff640 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d5804cf40 0x559d5804f330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ff09a5f3a8c950e server_cookie=e4e46ebb8eb5b19f in_seq=0 out_seq=0 2024-09-19T09:35:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.339+0000 7f0ce67fc640 10 client.25918 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:35:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.339+0000 7f0ce67fc640 1 -- 172.21.15.28:0/3165202489 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d57fa5860 con 0x559d5804cf40 2024-09-19T09:35:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.339+0000 7f0ce67fc640 10 client.25918 handle_command_reply: tid=0 2024-09-19T09:35:52.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.340+0000 7f0d076b0740 1 client.25918 shutdown 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.341+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 msgr2=0x559d5803cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.341+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001ab70 tx=0x7f0cf003b6d0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 shutdown_connections 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d5804cf40 0x559d5804f330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d58033b10 0x559d5803a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d5803d340 0x559d5803a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 --2- 172.21.15.28:0/3165202489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d580325a0 0x559d5803cd60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 >> 172.21.15.28:0/3165202489 conn(0x559d57ffb310 msgr2=0x559d58034900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0ce67fc640 0 client.25918 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:35:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.343+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 shutdown_connections 2024-09-19T09:35:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.345+0000 7f0d076b0740 1 -- 172.21.15.28:0/3165202489 wait complete. 2024-09-19T09:35:52.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.345+0000 7f0d076b0740 20 client.25918 trim_cache size 0 max 16384 2024-09-19T09:35:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.380+0000 7f0d076b0740 1 Processor -- start 2024-09-19T09:35:52.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.384+0000 7f0d076b0740 1 -- start start 2024-09-19T09:35:52.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 0x559d58039fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 0x559d580492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d5803d340 0x559d580350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001a2bd0 con 0x559d580325a0 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 0x559d580492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 0x559d580492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:54094/0 (socket says 172.21.15.28:54094) 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0cf7fff640 1 -- 172.21.15.28:0/2152112493 learned_addr learned my addr 172.21.15.28:0/2152112493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001a47d0 con 0x559d5803d340 2024-09-19T09:35:52.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.385+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001a32a0 con 0x559d58033b10 2024-09-19T09:35:52.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.388+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 0x559d58039fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 -- 172.21.15.28:0/2152112493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 msgr2=0x559d580492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 0x559d580492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 -- 172.21.15.28:0/2152112493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d5803d340 msgr2=0x559d580350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d5803d340 0x559d580350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 -- 172.21.15.28:0/2152112493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d58038560 con 0x559d580325a0 2024-09-19T09:35:52.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.389+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 0x559d58039fe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005fd0 tx=0x7f0cf00044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a011a0b54a6289f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.392+0000 7f0cf4ff9640 1 -- 172.21.15.28:0/2152112493 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0072020 con 0x559d580325a0 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.392+0000 7f0cf4ff9640 1 -- 172.21.15.28:0/2152112493 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf00054f0 con 0x559d580325a0 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.392+0000 7f0cf4ff9640 1 -- 172.21.15.28:0/2152112493 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf005e040 con 0x559d580325a0 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 msgr2=0x559d58039fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 0x559d58039fe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0005fd0 tx=0x7f0cf00044b0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 shutdown_connections 2024-09-19T09:35:52.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d5803d340 0x559d580350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d58033b10 0x559d580492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2152112493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d580325a0 0x559d58039fe0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.393+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 >> 172.21.15.28:0/2152112493 conn(0x559d57ffb310 msgr2=0x559d580398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.397+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 shutdown_connections 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.397+0000 7f0d076b0740 1 -- 172.21.15.28:0/2152112493 wait complete. 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.397+0000 7f0d076b0740 1 -- start start 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.397+0000 7f0d076b0740 1 Processor -- start 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.398+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d58045630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.398+0000 7f0d076b0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58045b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.398+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d58045630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d58045630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:40920/0 (socket says 172.21.15.28:40920) 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0cfeffd640 1 -- 172.21.15.28:0/2399099958 learned_addr learned my addr 172.21.15.28:0/2399099958 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d58036820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001a4040 con 0x559d58033b10 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001a3970 con 0x559d580325a0 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d58036820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.399+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000739a0 con 0x559d5803d340 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0d076b0740 10 client.0 _ll_register_callbacks cb 0x7f0cf0051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 -- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 msgr2=0x559d58045630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d58045630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 -- 172.21.15.28:0/2399099958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 msgr2=0x559d58045b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:52.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58045b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d58036f30 con 0x559d5803d340 2024-09-19T09:35:52.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0cff7fe640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d58036820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009fa0 tx=0x7f0cf8015e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757b9b1fb61df801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:52.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.400+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf00054f0 con 0x559d5803d340 2024-09-19T09:35:52.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.402+0000 7f0ce67fc640 10 client.25924 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:52.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.402+0000 7f0ce67fc640 1 -- 172.21.15.28:0/2399099958 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8003e40 con 0x559d5803d340 2024-09-19T09:35:52.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.402+0000 7f0ce67fc640 1 -- 172.21.15.28:0/2399099958 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f0cf8014420 con 0x559d5803d340 2024-09-19T09:35:52.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.402+0000 7f0ce67fc640 1 -- 172.21.15.28:0/2399099958 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0cf8014620 con 0x559d5803d340 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.403+0000 7f0d076b0740 10 client.25924 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.403+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=33}) -- 0x559d580b73e0 con 0x559d5803d340 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0ce67fc640 1 -- 172.21.15.28:0/2399099958 <== mon.1 v2:172.21.15.78:3300/0 4 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f0cf8025070 con 0x559d5803d340 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 10 client.25924 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 10 client.25924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 20 client.25924 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d580b79f0 0x559d580b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 4 client.25924 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:35:52.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.405+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 --> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d57f9fd70 con 0x559d580b79f0 2024-09-19T09:35:52.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.406+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d580b79f0 0x559d580b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.420+0000 7f0cfeffd640 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d580b79f0 0x559d580b9e50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c80957a21a8e9881 server_cookie=eacdc414aad4ee69 in_seq=0 out_seq=0 2024-09-19T09:35:52.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.422+0000 7f0ce67fc640 10 client.25924 ms_handle_connect on v2:172.21.15.38:6838/3114720816 2024-09-19T09:35:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.425+0000 7f0ce67fc640 1 -- 172.21.15.28:0/2399099958 <== mds.1 v2:172.21.15.38:6838/3114720816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d57f9fd70 con 0x559d580b79f0 2024-09-19T09:35:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.425+0000 7f0ce67fc640 10 client.25924 handle_command_reply: tid=0 2024-09-19T09:35:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.425+0000 7f0d076b0740 1 client.25924 shutdown 2024-09-19T09:35:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.430+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 msgr2=0x559d58036820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.430+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d58036820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009fa0 tx=0x7f0cf8015e70 comp rx=0 tx=0).stop 2024-09-19T09:35:52.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 shutdown_connections 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d5803d340 0x559d58036820 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d58033b10 0x559d58045b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:6838/3114720816,v1:172.21.15.38:6839/3114720816] conn(0x559d580b79f0 0x559d580b9e50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 --2- 172.21.15.28:0/2399099958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d580325a0 0x559d58045630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.431+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 >> 172.21.15.28:0/2399099958 conn(0x559d57ffb310 msgr2=0x559d5802d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.432+0000 7f0ce67fc640 0 client.25924 ms_handle_reset on v2:172.21.15.38:6838/3114720816 2024-09-19T09:35:52.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.439+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 shutdown_connections 2024-09-19T09:35:52.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.442+0000 7f0d076b0740 1 -- 172.21.15.28:0/2399099958 wait complete. 2024-09-19T09:35:52.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.447+0000 7f0d076b0740 20 client.25924 trim_cache size 0 max 16384 2024-09-19T09:35:52.452 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 msgr2=0x7f0cec0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 0x7f0cec0a5910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0cf001aac0 tx=0x7f0cf001a570 comp rx=0 tx=0).stop 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 msgr2=0x7f0d0010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800b450 tx=0x7f0ce800b910 comp rx=0 tx=0).stop 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 shutdown_connections 2024-09-19T09:35:52.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f0cec0a3450 0x7f0cec0a5910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00105ec0 0x7f0d00072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001011e0 0x7f0d0010dc70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 --2- 172.21.15.28:0/2035568322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d001008c0 0x7f0d0010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.454+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 >> 172.21.15.28:0/2035568322 conn(0x7f0d000fc7d0 msgr2=0x7f0d00103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.455+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 shutdown_connections 2024-09-19T09:35:52.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:52.455+0000 7f0d052d7640 1 -- 172.21.15.28:0/2035568322 wait complete. 2024-09-19T09:35:52.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:52 smithi078 ceph-mon[35619]: pgmap v434: 97 pgs: 97 active+clean; 222 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 616 op/s 2024-09-19T09:35:52.877 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:52 smithi028 ceph-mon[30972]: pgmap v434: 97 pgs: 97 active+clean; 222 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 616 op/s 2024-09-19T09:35:52.928 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:52 smithi038 ceph-mon[35271]: pgmap v434: 97 pgs: 97 active+clean; 222 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 616 op/s 2024-09-19T09:35:53.337 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:35:53.473 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T09:35:53.473 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:35:53.983 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:54.014 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:54.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:54 smithi028 ceph-mon[30972]: pgmap v435: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 386 op/s 2024-09-19T09:35:54.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:54 smithi078 ceph-mon[35619]: pgmap v435: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 386 op/s 2024-09-19T09:35:54.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:54 smithi038 ceph-mon[35271]: pgmap v435: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 386 op/s 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 -- 172.21.15.28:0/3424270933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d01001a0 msgr2=0x7f79d0100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/3424270933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d01001a0 0x7f79d0100580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009930 tx=0x7f79c0032c70 comp rx=0 tx=0).stop 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 -- 172.21.15.28:0/3424270933 shutdown_connections 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/3424270933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d0106cd0 0x7f79d0110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/3424270933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/3424270933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d01001a0 0x7f79d0100580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.135+0000 7f79d5ec9640 1 -- 172.21.15.28:0/3424270933 >> 172.21.15.28:0/3424270933 conn(0x7f79d0077070 msgr2=0x7f79d00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.140+0000 7f79d5ec9640 1 -- 172.21.15.28:0/3424270933 shutdown_connections 2024-09-19T09:35:55.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.145+0000 7f79d5ec9640 1 -- 172.21.15.28:0/3424270933 wait complete. 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.145+0000 7f79d5ec9640 1 Processor -- start 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.145+0000 7f79d5ec9640 1 -- start start 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 0x7f79d01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d4ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 0x7f79d01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d4ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 0x7f79d01965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35500/0 (socket says 172.21.15.28:35500) 2024-09-19T09:35:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d4ec7640 1 -- 172.21.15.28:0/694313216 learned_addr learned my addr 172.21.15.28:0/694313216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:55.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d0106cd0 0x7f79d019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010b240 con 0x7f79d0100ac0 2024-09-19T09:35:55.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010b0c0 con 0x7f79d0106cd0 2024-09-19T09:35:55.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010b3c0 con 0x7f79d01001a0 2024-09-19T09:35:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 -- 172.21.15.28:0/694313216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 msgr2=0x7f79d01965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 0x7f79d01965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 -- 172.21.15.28:0/694313216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d0106cd0 msgr2=0x7f79d019a970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d0106cd0 0x7f79d019a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.150+0000 7f79cffff640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d019b050 con 0x7f79d0100ac0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79cffff640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0196b00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f79c400e770 tx=0x7f79c400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79cdffb640 1 -- 172.21.15.28:0/694313216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c400ccc0 con 0x7f79d0100ac0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79cdffb640 1 -- 172.21.15.28:0/694313216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c4009530 con 0x7f79d0100ac0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79cdffb640 1 -- 172.21.15.28:0/694313216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4010430 con 0x7f79d0100ac0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d019b340 con 0x7f79d0100ac0 2024-09-19T09:35:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.155+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01a2c20 con 0x7f79d0100ac0 2024-09-19T09:35:55.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.159+0000 7f79cdffb640 1 -- 172.21.15.28:0/694313216 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f79c4003f00 con 0x7f79d0100ac0 2024-09-19T09:35:55.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.159+0000 7f79cdffb640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 0x7f79b80a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79cdffb640 1 -- 172.21.15.28:0/694313216 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f79c409a360 con 0x7f79d0100ac0 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 0x7f79b80a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 Processor -- start 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 -- start start 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b470127b0 0x557b47012bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 0x557b4701f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b46f782f0 con 0x557b470127b0 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.164+0000 7f79d9504740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557b46f1fcf0 con 0x557b4701d550 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d9504740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b46f742b0 con 0x557b47013d20 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35512/0 (socket says 172.21.15.28:35512) 2024-09-19T09:35:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3382657667 learned_addr learned my addr 172.21.15.28:0/3382657667 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d56c8640 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 0x557b4701f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3382657667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 msgr2=0x557b4701f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 0x557b4701f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3382657667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b470127b0 msgr2=0x557b47012bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b470127b0 0x557b47012bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3382657667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b46f043a0 con 0x557b47013d20 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004380 tx=0x7f79c00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c29ac96b7954dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d56c8640 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 0x557b4701f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 0x7f79b80a5890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79c0049000 tx=0x7f79c0003630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.169+0000 7f79b6ffd640 1 -- 172.21.15.28:0/3382657667 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0033a50 con 0x557b47013d20 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.174+0000 7f79b6ffd640 1 -- 172.21.15.28:0/3382657667 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0033bf0 con 0x557b47013d20 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.174+0000 7f79d9504740 1 -- 172.21.15.28:0/3382657667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 msgr2=0x557b47012270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.174+0000 7f79d9504740 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004380 tx=0x7f79c00043b0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.178+0000 7f79d9504740 1 -- 172.21.15.28:0/3382657667 shutdown_connections 2024-09-19T09:35:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.178+0000 7f79d9504740 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b4701d550 0x557b4701f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.178+0000 7f79d9504740 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b470127b0 0x557b47012bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.178+0000 7f79d9504740 1 --2- 172.21.15.28:0/3382657667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b47013d20 0x557b47012270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.178+0000 7f79d9504740 1 -- 172.21.15.28:0/3382657667 >> 172.21.15.28:0/3382657667 conn(0x557b46fdb310 msgr2=0x557b4700ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.179+0000 7f79d9504740 1 -- 172.21.15.28:0/3382657667 shutdown_connections 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.179+0000 7f79d9504740 1 -- 172.21.15.28:0/3382657667 wait complete. 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.179+0000 7f79d9504740 1 -- start start 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.179+0000 7f79d9504740 1 Processor -- start 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.180+0000 7f79d9504740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b4701ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b4701a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b4701a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b47031250 con 0x557b47013d20 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557b46f33060 con 0x557b470127b0 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b470313d0 con 0x557b4701d550 2024-09-19T09:35:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.181+0000 7f79d9504740 10 client.0 _ll_register_callbacks cb 0x557b46c868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:55.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.188+0000 7f79cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b4701a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.188+0000 7f79cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b4701a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55212/0 (socket says 172.21.15.28:55212) 2024-09-19T09:35:55.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.188+0000 7f79cffff640 1 -- 172.21.15.28:0/3898547155 learned_addr learned my addr 172.21.15.28:0/3898547155 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:55.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.188+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b4701ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.188+0000 7f79d56c8640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b4701a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.191+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 msgr2=0x557b4701a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.191+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b4701a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.191+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3898547155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 msgr2=0x557b4701a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.191+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b4701a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.191+0000 7f79d4ec7640 1 -- 172.21.15.28:0/3898547155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b46ce93c0 con 0x557b470127b0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.192+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b4701ce90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004380 tx=0x7f79c0032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dda756e8b42cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.192+0000 7f79affff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.192+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00146a0 con 0x557b470127b0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.192+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f79c0014840 con 0x557b470127b0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.194+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c0033bf0 con 0x557b470127b0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.196+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79c00046c0 con 0x557b470127b0 2024-09-19T09:35:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.196+0000 7f79d9504740 10 client.25973 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.196+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557b4703cc00 con 0x557b470127b0 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.197+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0033bf0 con 0x557b470127b0 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.197+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f79c00644e0 con 0x557b470127b0 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 10 client.25973 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 20 client.25973 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b4703df50 0x557b47040340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 4 client.25973 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b46f85860 con 0x557b4703df50 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.198+0000 7f79cffff640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b4703df50 0x557b47040340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.199+0000 7f79affff640 10 client.25973 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.200+0000 7f79cffff640 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b4703df50 0x557b47040340 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8e3a0a86d1e135 server_cookie=bf9458a9ca4370d8 in_seq=0 out_seq=0 2024-09-19T09:35:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79affff640 1 -- 172.21.15.28:0/3898547155 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f79c40abe80 con 0x557b4703df50 2024-09-19T09:35:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79affff640 10 client.25973 handle_command_reply: tid=0 2024-09-19T09:35:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 client.25973 shutdown 2024-09-19T09:35:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 msgr2=0x557b4701ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b4701ce90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004380 tx=0x7f79c0032c70 comp rx=0 tx=0).stop 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 shutdown_connections 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b4703df50 0x557b47040340 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b4701a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b4701a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 --2- 172.21.15.28:0/3898547155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b4701ce90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.206+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 >> 172.21.15.28:0/3898547155 conn(0x557b46fdb310 msgr2=0x557b47014b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.207+0000 7f79affff640 0 client.25973 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.207+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 shutdown_connections 2024-09-19T09:35:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.207+0000 7f79d9504740 1 -- 172.21.15.28:0/3898547155 wait complete. 2024-09-19T09:35:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.207+0000 7f79d9504740 20 client.25973 trim_cache size 0 max 16384 2024-09-19T09:35:55.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.245+0000 7f79d9504740 1 Processor -- start 2024-09-19T09:35:55.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.245+0000 7f79d9504740 1 -- start start 2024-09-19T09:35:55.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.247+0000 7f79d9504740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b470127b0 0x557b4703a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.247+0000 7f79d9504740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.247+0000 7f79d9504740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4701d550 0x557b47015300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.247+0000 7f79d9504740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01ab1a0 con 0x557b4701d550 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79d9504740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01acda0 con 0x557b47013d20 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79d9504740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01ab870 con 0x557b470127b0 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79d4ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b470127b0 0x557b4703a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:52768/0 (socket says 172.21.15.28:52768) 2024-09-19T09:35:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.248+0000 7f79cffff640 1 -- 172.21.15.28:0/2317223510 learned_addr learned my addr 172.21.15.28:0/2317223510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.249+0000 7f79cffff640 1 -- 172.21.15.28:0/2317223510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b470127b0 msgr2=0x557b4703a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.249+0000 7f79cffff640 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b470127b0 0x557b4703a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.249+0000 7f79cffff640 1 -- 172.21.15.28:0/2317223510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4701d550 msgr2=0x557b47015300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.249+0000 7f79cffff640 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4701d550 0x557b47015300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.249+0000 7f79cffff640 1 -- 172.21.15.28:0/2317223510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4701d200 con 0x557b47013d20 2024-09-19T09:35:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.250+0000 7f79cffff640 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79c400c6a0 tx=0x7f79c4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a547822d5e4b5ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.251+0000 7f79b67fc640 1 -- 172.21.15.28:0/2317223510 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40ba070 con 0x557b47013d20 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.251+0000 7f79b67fc640 1 -- 172.21.15.28:0/2317223510 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b46f85860 con 0x557b47013d20 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.251+0000 7f79b67fc640 1 -- 172.21.15.28:0/2317223510 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40cd8b0 con 0x557b47013d20 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.252+0000 7f79d9504740 1 -- 172.21.15.28:0/2317223510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 msgr2=0x557b4703ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.252+0000 7f79d9504740 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79c400c6a0 tx=0x7f79c4004710 comp rx=0 tx=0).stop 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.253+0000 7f79d9504740 1 -- 172.21.15.28:0/2317223510 shutdown_connections 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.253+0000 7f79d9504740 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b47013d20 0x557b4703ff60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.253+0000 7f79d9504740 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4701d550 0x557b47015300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.253+0000 7f79d9504740 1 --2- 172.21.15.28:0/2317223510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b470127b0 0x557b4703a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.253+0000 7f79d9504740 1 -- 172.21.15.28:0/2317223510 >> 172.21.15.28:0/2317223510 conn(0x557b46fdb310 msgr2=0x557b4700dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.256+0000 7f79d9504740 1 -- 172.21.15.28:0/2317223510 shutdown_connections 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.256+0000 7f79d9504740 1 -- 172.21.15.28:0/2317223510 wait complete. 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.256+0000 7f79d9504740 1 -- start start 2024-09-19T09:35:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.257+0000 7f79d9504740 1 Processor -- start 2024-09-19T09:35:55.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.258+0000 7f79d9504740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b47036d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.258+0000 7f79d9504740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b47032090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d9504740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b470325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d9504740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01ac610 con 0x557b47013d20 2024-09-19T09:35:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d9504740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01abf40 con 0x557b470127b0 2024-09-19T09:35:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d56c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b470325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d56c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b470325d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35544/0 (socket says 172.21.15.28:35544) 2024-09-19T09:35:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d56c8640 1 -- 172.21.15.28:0/2776656183 learned_addr learned my addr 172.21.15.28:0/2776656183 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b47036d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01a9180 con 0x557b4701d550 2024-09-19T09:35:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.259+0000 7f79d9504740 10 client.0 _ll_register_callbacks cb 0x7f79c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:55.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.262+0000 7f79d4ec7640 1 -- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 msgr2=0x557b470325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.262+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b470325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.262+0000 7f79d4ec7640 1 -- 172.21.15.28:0/2776656183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 msgr2=0x557b47032090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.262+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b47032090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.262+0000 7f79d4ec7640 1 -- 172.21.15.28:0/2776656183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b47032dd0 con 0x557b470127b0 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.266+0000 7f79d4ec7640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b47036d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79c003d040 tx=0x7f79c0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29e6044ed0e8b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.266+0000 7f79affff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.266+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c006c070 con 0x557b470127b0 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f79c0064a30 con 0x557b470127b0 2024-09-19T09:35:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b46f85860 con 0x557b470127b0 2024-09-19T09:35:55.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79c0078070 con 0x557b470127b0 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79d9504740 10 client.25979 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557b470a8910 con 0x557b470127b0 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.267+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b46f85860 con 0x557b470127b0 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f79c0005090 con 0x557b470127b0 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 10 client.25979 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 10 client.25979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 20 client.25979 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b470a9090 0x557b470ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 4 client.25979 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:35:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.268+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b46f7fd70 con 0x557b470a9090 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.271+0000 7f79cffff640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b470a9090 0x557b470ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.271+0000 7f79affff640 10 client.25979 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.272+0000 7f79cffff640 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b470a9090 0x557b470ab4b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1300149a8891594d server_cookie=45783eca1ce4d23f in_seq=0 out_seq=0 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.279+0000 7f79affff640 1 -- 172.21.15.28:0/2776656183 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b46f7fd70 con 0x557b470a9090 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.279+0000 7f79affff640 10 client.25979 handle_command_reply: tid=0 2024-09-19T09:35:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.279+0000 7f79d9504740 1 client.25979 shutdown 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.283+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 msgr2=0x557b47036d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.283+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b47036d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79c003d040 tx=0x7f79c0009260 comp rx=0 tx=0).stop 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 shutdown_connections 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557b470a9090 0x557b470ab4b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4701d550 0x557b470325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b47013d20 0x557b47032090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 --2- 172.21.15.28:0/2776656183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557b470127b0 0x557b47036d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 >> 172.21.15.28:0/2776656183 conn(0x557b46fdb310 msgr2=0x557b4701b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79affff640 0 client.25979 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 shutdown_connections 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.286+0000 7f79d9504740 1 -- 172.21.15.28:0/2776656183 wait complete. 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.287+0000 7f79d9504740 20 client.25979 trim_cache size 0 max 16384 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.295+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 msgr2=0x7f79b80a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.295+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 0x7f79b80a5890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79c0049000 tx=0x7f79c0003630 comp rx=0 tx=0).stop 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.295+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 msgr2=0x7f79d0196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.295+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0196b00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f79c400e770 tx=0x7f79c400ec30 comp rx=0 tx=0).stop 2024-09-19T09:35:55.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 shutdown_connections 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f79b80a33d0 0x7f79b80a5890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d0106cd0 0x7f79d019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f79d0100ac0 0x7f79d0196b00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 --2- 172.21.15.28:0/694313216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d01001a0 0x7f79d01965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 >> 172.21.15.28:0/694313216 conn(0x7f79d0077070 msgr2=0x7f79d0105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 shutdown_connections 2024-09-19T09:35:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.297+0000 7f79d5ec9640 1 -- 172.21.15.28:0/694313216 wait complete. 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"btime":"2024-09-19T09:35:21:300725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":25804,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:35:21.216576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 -- 172.21.15.28:0/649231731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 msgr2=0x7f821011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 --2- 172.21.15.28:0/649231731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f821011d530 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8200009900 tx=0x7f820002a440 comp rx=0 tx=0).stop 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 -- 172.21.15.28:0/649231731 shutdown_connections 2024-09-19T09:35:56.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 --2- 172.21.15.28:0/649231731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f821011d530 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 --2- 172.21.15.28:0/649231731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f821006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 --2- 172.21.15.28:0/649231731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8210110190 0x7f821006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.609+0000 7f821749e640 1 -- 172.21.15.28:0/649231731 >> 172.21.15.28:0/649231731 conn(0x7f8210073180 msgr2=0x7f8210073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.610+0000 7f821749e640 1 -- 172.21.15.28:0/649231731 shutdown_connections 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.610+0000 7f821749e640 1 -- 172.21.15.28:0/649231731 wait complete. 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.611+0000 7f821749e640 1 Processor -- start 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.611+0000 7f821749e640 1 -- start start 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8210110190 0x7f82101a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f82101abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8210120c90 con 0x7f821011b140 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8210120b10 con 0x7f8210110190 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821749e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8210120e10 con 0x7f821006ea30 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f8216c9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f82101abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35568/0 (socket says 172.21.15.28:35568) 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 -- 172.21.15.28:0/1606021865 learned_addr learned my addr 172.21.15.28:0/1606021865 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 -- 172.21.15.28:0/1606021865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8210110190 msgr2=0x7f82101a77d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8210110190 0x7f82101a77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 -- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 msgr2=0x7f82101abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f82101abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.612+0000 7f821649c640 1 -- 172.21.15.28:0/1606021865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82101ac240 con 0x7f821006ea30 2024-09-19T09:35:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f821649c640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f820800ba50 tx=0x7f820800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208019070 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82101ac530 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820800cc10 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080076a0 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.613+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82101b3e10 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.615+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821006eef0 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.616+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f820801c070 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.617+0000 7f82077fe640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 0x7f81fc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.617+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f82080cee80 con 0x7f821006ea30 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.617+0000 7f8215c9b640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 0x7f81fc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.618+0000 7f8215c9b640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 0x7f81fc0a57c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f821011ad10 tx=0x7f820c006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:56.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.621+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f820809b580 con 0x7f821006ea30 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.951+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f821006dc50 con 0x7f821006ea30 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.953+0000 7f82077fe640 1 -- 172.21.15.28:0/1606021865 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11957 (secure 0 0 0) 0x7f820809f880 con 0x7f821006ea30 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.957+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 msgr2=0x7f81fc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.957+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 0x7f81fc0a57c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f821011ad10 tx=0x7f820c006bd0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.957+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 msgr2=0x7f82101a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.957+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f820800ba50 tx=0x7f820800bf10 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 shutdown_connections 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f81fc0a3300 0x7f81fc0a57c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8210110190 0x7f82101a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f821011b140 0x7f82101abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 --2- 172.21.15.28:0/1606021865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f821006ea30 0x7f82101a7290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:56.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.958+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 >> 172.21.15.28:0/1606021865 conn(0x7f8210073180 msgr2=0x7f8210119460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:56.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.961+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 shutdown_connections 2024-09-19T09:35:56.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:55.962+0000 7f821749e640 1 -- 172.21.15.28:0/1606021865 wait complete. 2024-09-19T09:35:56.524 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 43505211, 'ino': 1099511635020, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml'}, {'damage_type': 'backtrace', 'id': 61911944, 'ino': 1099511635273, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 115109522, 'ino': 1099511635268, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 183892634, 'ino': 1099511635078, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-38.xml'}, {'damage_type': 'backtrace', 'id': 240764108, 'ino': 1099511635258, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml'}, {'damage_type': 'backtrace', 'id': 338749725, 'ino': 1099511635328, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 400686254, 'ino': 1099511635062, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml'}, {'damage_type': 'backtrace', 'id': 446759179, 'ino': 1099511635035, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml'}, {'damage_type': 'backtrace', 'id': 462664256, 'ino': 1099511635357, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 479171130, 'ino': 1099511635248, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-4.xml'}, {'damage_type': 'backtrace', 'id': 504655425, 'ino': 1099511635322, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 513296571, 'ino': 1099511635042, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml'}, {'damage_type': 'backtrace', 'id': 549348820, 'ino': 1099511635399, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 755790205, 'ino': 1099511635098, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml'}, {'damage_type': 'backtrace', 'id': 761139812, 'ino': 1099511635017, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-55.xml'}, {'damage_type': 'backtrace', 'id': 762961691, 'ino': 1099511635058, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml'}, {'damage_type': 'backtrace', 'id': 777647174, 'ino': 1099511635014, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml'}, {'damage_type': 'backtrace', 'id': 789521767, 'ino': 1099511635083, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml'}, {'damage_type': 'backtrace', 'id': 822673586, 'ino': 1099511635038, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml'}, {'damage_type': 'backtrace', 'id': 826824813, 'ino': 1099511635361, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 831921796, 'ino': 1099511635010, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml'}, {'damage_type': 'backtrace', 'id': 865945515, 'ino': 1099511635263, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml'}, {'damage_type': 'backtrace', 'id': 866826736, 'ino': 1099511635103, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-33.xml'}, {'damage_type': 'backtrace', 'id': 916280144, 'ino': 1099511635337, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 948788196, 'ino': 1099511635395, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 972844044, 'ino': 1099511635316, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 995828104, 'ino': 1099511635093, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-35.xml'}, {'damage_type': 'backtrace', 'id': 1048844883, 'ino': 1099511635050, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml'}, {'damage_type': 'backtrace', 'id': 1078908937, 'ino': 1099511635310, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 1110953659, 'ino': 1099511635133, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml'}, {'damage_type': 'backtrace', 'id': 1111906343, 'ino': 1099511635238, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml'}, {'damage_type': 'backtrace', 'id': 1154068137, 'ino': 1099511635046, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml'}, {'damage_type': 'backtrace', 'id': 1160714455, 'ino': 1099511635158, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml'}, {'damage_type': 'backtrace', 'id': 1181845281, 'ino': 1099511635193, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml'}, {'damage_type': 'backtrace', 'id': 1245952906, 'ino': 1099511635029, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-51.xml'}, {'damage_type': 'backtrace', 'id': 1262330276, 'ino': 1099511635183, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml'}, {'damage_type': 'backtrace', 'id': 1273027543, 'ino': 1099511635218, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml'}, {'damage_type': 'backtrace', 'id': 1296281627, 'ino': 1099511635148, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml'}, {'damage_type': 'backtrace', 'id': 1478194775, 'ino': 1099511635379, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 1478222328, 'ino': 1099511635001, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml'}, {'damage_type': 'backtrace', 'id': 1486367090, 'ino': 1099511635253, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-3.xml'}, {'damage_type': 'backtrace', 'id': 1532872448, 'ino': 1099511635143, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml'}, {'damage_type': 'backtrace', 'id': 1545842105, 'ino': 1099511635384, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 1549326968, 'ino': 1099511635341, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 1570516012, 'ino': 1099511635304, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 1589658122, 'ino': 1099511635208, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1593190922, 'ino': 1099511635163, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml'}, {'damage_type': 'backtrace', 'id': 1594183362, 'ino': 1099511635298, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 1776254104, 'ino': 1099511635333, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 1778768791, 'ino': 1099511635032, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-50.xml'}, {'damage_type': 'backtrace', 'id': 1814649507, 'ino': 1099511635198, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml'}, {'damage_type': 'backtrace', 'id': 1953341074, 'ino': 1099511635074, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml'}, {'damage_type': 'backtrace', 'id': 2079664567, 'ino': 1099511635233, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml'}, {'damage_type': 'backtrace', 'id': 2095444442, 'ino': 1099511635004, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml'}, {'damage_type': 'backtrace', 'id': 2142132056, 'ino': 1099511635123, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2335563959, 'ino': 1099511635203, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml'}, {'damage_type': 'backtrace', 'id': 2440389343, 'ino': 1099511635228, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml'}, {'damage_type': 'backtrace', 'id': 2462381569, 'ino': 1099511635369, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 2473727998, 'ino': 1099511635153, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml'}, {'damage_type': 'backtrace', 'id': 2527562726, 'ino': 1099511635128, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml'}, {'damage_type': 'backtrace', 'id': 2620378249, 'ino': 1099511635345, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 2802340230, 'ino': 1099511635108, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml'}, {'damage_type': 'backtrace', 'id': 2838364705, 'ino': 1099511635173, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml'}, {'damage_type': 'backtrace', 'id': 2918815820, 'ino': 1099511635188, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml'}, {'damage_type': 'backtrace', 'id': 2941740507, 'ino': 1099511635292, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 2968186054, 'ino': 1099511635374, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 2993810965, 'ino': 1099511635286, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 3036187249, 'ino': 1099511635113, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml'}, {'damage_type': 'backtrace', 'id': 3209832212, 'ino': 1099511635088, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml'}, {'damage_type': 'backtrace', 'id': 3243746793, 'ino': 1099511635178, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml'}, {'damage_type': 'backtrace', 'id': 3421895188, 'ino': 1099511635070, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml'}, {'damage_type': 'backtrace', 'id': 3463495752, 'ino': 1099511634999, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml'}, {'damage_type': 'backtrace', 'id': 3509476447, 'ino': 1099511635026, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml'}, {'damage_type': 'backtrace', 'id': 3526153903, 'ino': 1099511635390, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 3531836474, 'ino': 1099511635280, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 3542167398, 'ino': 1099511635365, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 3673562544, 'ino': 1099511635353, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3703137601, 'ino': 1099511635066, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml'}, {'damage_type': 'backtrace', 'id': 3758349876, 'ino': 1099511635213, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml'}, {'damage_type': 'backtrace', 'id': 3778975836, 'ino': 1099511635023, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml'}, {'damage_type': 'backtrace', 'id': 3909603654, 'ino': 1099511635138, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml'}, {'damage_type': 'backtrace', 'id': 3930130926, 'ino': 1099511635054, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml'}, {'damage_type': 'backtrace', 'id': 3937188258, 'ino': 1099511635223, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml'}, {'damage_type': 'backtrace', 'id': 3972249198, 'ino': 1099511635168, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml'}, {'damage_type': 'backtrace', 'id': 3993397751, 'ino': 1099511635118, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-30.xml'}, {'damage_type': 'backtrace', 'id': 4094002322, 'ino': 1099511635243, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml'}, {'damage_type': 'backtrace', 'id': 4174800544, 'ino': 1099511635008, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml'}, {'damage_type': 'backtrace', 'id': 4290209847, 'ino': 1099511635349, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg'}] 2024-09-19T09:35:56.524 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 64214544, 'ino': 1099511635501, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-12.xml'}, {'damage_type': 'backtrace', 'id': 78899554, 'ino': 1099511635469, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-20.xml'}, {'damage_type': 'backtrace', 'id': 141973061, 'ino': 1099511635513, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-9.xml'}, {'damage_type': 'backtrace', 'id': 146550252, 'ino': 1099511635497, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-13.xml'}, {'damage_type': 'backtrace', 'id': 189050940, 'ino': 1099511635489, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-15.xml'}, {'damage_type': 'backtrace', 'id': 292116881, 'ino': 1099511635521, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-7.xml'}, {'damage_type': 'backtrace', 'id': 293957799, 'ino': 1099511635461, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-22.xml'}, {'damage_type': 'backtrace', 'id': 746072299, 'ino': 1099511635427, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-29.xml'}, {'damage_type': 'backtrace', 'id': 786826450, 'ino': 1099511635442, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-26.xml'}, {'damage_type': 'backtrace', 'id': 810526816, 'ino': 1099511635432, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-28.xml'}, {'damage_type': 'backtrace', 'id': 837056959, 'ino': 1099511635422, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-30.xml'}, {'damage_type': 'backtrace', 'id': 1154397641, 'ino': 1099511635517, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-8.xml'}, {'damage_type': 'backtrace', 'id': 1184106603, 'ino': 1099511635452, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-24.xml'}, {'damage_type': 'backtrace', 'id': 1185431172, 'ino': 1099511635509, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-10.xml'}, {'damage_type': 'backtrace', 'id': 1322583734, 'ino': 1099511635537, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-3.xml'}, {'damage_type': 'backtrace', 'id': 1509241879, 'ino': 1099511635485, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-16.xml'}, {'damage_type': 'backtrace', 'id': 1610455798, 'ino': 1099511635505, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-11.xml'}, {'damage_type': 'backtrace', 'id': 1708148825, 'ino': 1099511635412, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-32.xml'}, {'damage_type': 'backtrace', 'id': 1872315549, 'ino': 1099511635525, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-6.xml'}, {'damage_type': 'backtrace', 'id': 2168583767, 'ino': 1099511635407, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-33.xml'}, {'damage_type': 'backtrace', 'id': 2304095153, 'ino': 1099511635477, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml'}, {'damage_type': 'backtrace', 'id': 2476168433, 'ino': 1099511635541, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-2.xml'}, {'damage_type': 'backtrace', 'id': 2611167917, 'ino': 1099511635457, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-23.xml'}, {'damage_type': 'backtrace', 'id': 2923359205, 'ino': 1099511635437, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-27.xml'}, {'damage_type': 'backtrace', 'id': 2987978973, 'ino': 1099511635465, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-21.xml'}, {'damage_type': 'backtrace', 'id': 3204859604, 'ino': 1099511635533, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-4.xml'}, {'damage_type': 'backtrace', 'id': 3490786680, 'ino': 1099511635473, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-19.xml'}, {'damage_type': 'backtrace', 'id': 3734261190, 'ino': 1099511635529, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-5.xml'}, {'damage_type': 'backtrace', 'id': 3907198819, 'ino': 1099511635493, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-14.xml'}, {'damage_type': 'backtrace', 'id': 4042894018, 'ino': 1099511635447, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-25.xml'}, {'damage_type': 'backtrace', 'id': 4102236919, 'ino': 1099511635481, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-17.xml'}, {'damage_type': 'backtrace', 'id': 4187979856, 'ino': 1099511635417, 'path': '/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-31.xml'}] 2024-09-19T09:35:56.524 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-09-19T09:35:56.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:56 smithi028 ceph-mon[30972]: pgmap v436: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 272 op/s 2024-09-19T09:35:56.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:56 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1606021865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:56.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:56 smithi078 ceph-mon[35619]: pgmap v436: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 272 op/s 2024-09-19T09:35:56.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:56 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1606021865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:56.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:56 smithi038 ceph-mon[35271]: pgmap v436: 97 pgs: 97 active+clean; 225 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 272 op/s 2024-09-19T09:35:56.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:56 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1606021865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:35:56.984 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T09:35:56.984 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T09:35:57.242 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:35:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.288+0000 7fcd39ffb640 1 -- 172.21.15.28:0/1313042839 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fddffa2a20 con 0x55fde00bba40 2024-09-19T09:35:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.288+0000 7fcd39ffb640 10 client.25912 handle_command_reply: tid=0 2024-09-19T09:35:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.288+0000 7fcd5b262740 1 client.25912 shutdown 2024-09-19T09:35:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 msgr2=0x55fde0049200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00b750 tx=0x7fcd4c004370 comp rx=0 tx=0).stop 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 shutdown_connections 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fde00322d0 0x55fde0049740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55fde00bba40 0x55fde00bde60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fde0037360 0x55fde004a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 --2- 172.21.15.28:0/1313042839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fde0031990 0x55fde0049200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd39ffb640 0 client.25912 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 >> 172.21.15.28:0/1313042839 conn(0x55fde002d000 msgr2=0x55fde002c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 shutdown_connections 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 1 -- 172.21.15.28:0/1313042839 wait complete. 2024-09-19T09:35:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.289+0000 7fcd5b262740 20 client.25912 trim_cache size 0 max 16384 2024-09-19T09:35:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:35:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:35:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:35:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 msgr2=0x7fcd3c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 0x7fcd3c0a5800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009b50 tx=0x7fcd40010040 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 msgr2=0x7fcd5019e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009430 tx=0x7fcd480319f0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 shutdown_connections 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcd3c0a3340 0x7fcd3c0a5800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd50101c20 0x7fcd50197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd501025e0 0x7fcd5019e2b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 --2- 172.21.15.28:0/262685471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd50100a30 0x7fcd50196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 >> 172.21.15.28:0/262685471 conn(0x7fcd500fc190 msgr2=0x7fcd50104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 shutdown_connections 2024-09-19T09:35:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:35:57.290+0000 7fcd583e8640 1 -- 172.21.15.28:0/262685471 wait complete. 2024-09-19T09:35:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 -- 172.21.15.28:0/889729372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 msgr2=0x7fa8ac1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/889729372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1067f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009900 tx=0x7fa8a002a410 comp rx=0 tx=0).stop 2024-09-19T09:35:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 -- 172.21.15.28:0/889729372 shutdown_connections 2024-09-19T09:35:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/889729372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac106f20 0x7fa8ac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/889729372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1067f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/889729372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac101df0 0x7fa8ac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 -- 172.21.15.28:0/889729372 >> 172.21.15.28:0/889729372 conn(0x7fa8ac0fd8f0 msgr2=0x7fa8ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 -- 172.21.15.28:0/889729372 shutdown_connections 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.752+0000 7fa8b12c1640 1 -- 172.21.15.28:0/889729372 wait complete. 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.753+0000 7fa8b12c1640 1 Processor -- start 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.753+0000 7fa8b12c1640 1 -- start start 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac101df0 0x7fa8ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 0x7fa8ac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111f40 con 0x7fa8ac1027a0 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111dc0 con 0x7fa8ac101df0 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8b12c1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1120c0 con 0x7fa8ac106f20 2024-09-19T09:35:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55264/0 (socket says 172.21.15.28:55264) 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/2090737232 learned_addr learned my addr 172.21.15.28:0/2090737232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 0x7fa8ac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac101df0 0x7fa8ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 -- 172.21.15.28:0/2090737232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac101df0 msgr2=0x7fa8ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac101df0 0x7fa8ac198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 -- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 msgr2=0x7fa8ac1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 -- 172.21.15.28:0/2090737232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac19d430 con 0x7fa8ac106f20 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.754+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 0x7fa8ac19cd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009fd0 tx=0x7fa8a400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.755+0000 7fa893fff640 1 -- 172.21.15.28:0/2090737232 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400ce20 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.755+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac19d720 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.755+0000 7fa893fff640 1 -- 172.21.15.28:0/2090737232 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a4005020 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.755+0000 7fa893fff640 1 -- 172.21.15.28:0/2090737232 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4010720 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.755+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac074420 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.757+0000 7fa8b369a740 1 Processor -- start 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.757+0000 7fa893fff640 1 -- 172.21.15.28:0/2090737232 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa8a4005540 con 0x7fa8ac106f20 2024-09-19T09:35:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.757+0000 7fa8b369a740 1 -- start start 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcc7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fcd2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a27fc2d2a0 con 0x55a27fcd2940 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a27fbd4b10 con 0x55a27fcc8080 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8b369a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a27fc29260 con 0x55a27fcc7760 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcc7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcc7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35596/0 (socket says 172.21.15.28:35596) 2024-09-19T09:35:57.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/1196577894 learned_addr learned my addr 172.21.15.28:0/1196577894 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fcd2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa893fff640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 0x7fa8940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1196577894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 msgr2=0x55a27fcc7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcc7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1196577894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 msgr2=0x55a27fcd4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 0x7fa8940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1196577894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27f99eb90 con 0x55a27fcc8080 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.758+0000 7fa893fff640 1 -- 172.21.15.28:0/2090737232 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa8a40cff00 con 0x7fa8ac106f20 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.759+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fcd2400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c20 tx=0x7fa89c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5087749a996b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.759+0000 7fa8917fa640 1 -- 172.21.15.28:0/1196577894 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c01e070 con 0x55a27fcc8080 2024-09-19T09:35:57.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.759+0000 7fa8917fa640 1 -- 172.21.15.28:0/1196577894 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c009dd0 con 0x55a27fcc8080 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- 172.21.15.28:0/1196577894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 msgr2=0x55a27fcd2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8ab7fe640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 0x7fa8940a5780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8a409c250 tx=0x7fa8a4066f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fcd2400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c20 tx=0x7fa89c011810 comp rx=0 tx=0).stop 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8917fa640 1 -- 172.21.15.28:0/1196577894 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0187a0 con 0x55a27fcc8080 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- 172.21.15.28:0/1196577894 shutdown_connections 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fcd2400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1196577894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcc7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- 172.21.15.28:0/1196577894 >> 172.21.15.28:0/1196577894 conn(0x55a27fcc3cf0 msgr2=0x55a27fcc40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- 172.21.15.28:0/1196577894 shutdown_connections 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- 172.21.15.28:0/1196577894 wait complete. 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 -- start start 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.760+0000 7fa8b369a740 1 Processor -- start 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fccf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fccfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a27fcd55b0 con 0x55a27fcd2940 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a27fbb56e0 con 0x55a27fcc8080 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a27fcd5730 con 0x55a27fcc7760 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8b369a740 10 client.0 _ll_register_callbacks cb 0x55a27fbd9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55274/0 (socket says 172.21.15.28:55274) 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/1226795348 learned_addr learned my addr 172.21.15.28:0/1226795348 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fccf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fccfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 msgr2=0x55a27fccf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fccf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/1226795348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 msgr2=0x55a27fccfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fccfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aa7fc640 1 -- 172.21.15.28:0/1226795348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a27fcdea70 con 0x55a27fcd2940 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.761+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fccfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00057c0 tx=0x7fa8a0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1be608a4ca3ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0035070 con 0x55a27fcd2940 2024-09-19T09:35:57.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c009dd0 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa8a0004030 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8a0052050 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8b369a740 10 client.16407 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a27fce1390 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c009dd0 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fa8a00047e0 con 0x55a27fcd2940 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8b369a740 10 client.16407 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.762+0000 7fa8b369a740 10 client.16407 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8b369a740 20 client.16407 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fce27a0 0x55a27fce4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8b369a740 4 client.16407 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a27fc3a9d0 con 0x55a27fce27a0 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fce27a0 0x55a27fce4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fce27a0 0x55a27fce4be0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5735e1dbcc0f54 server_cookie=ce70076700314a4 in_seq=0 out_seq=0 2024-09-19T09:35:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.763+0000 7fa88eff5640 10 client.16407 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.765+0000 7fa88eff5640 1 -- 172.21.15.28:0/1226795348 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a27fc3a9d0 con 0x55a27fce27a0 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.765+0000 7fa88eff5640 10 client.16407 handle_command_reply: tid=0 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.765+0000 7fa8b369a740 1 client.16407 shutdown 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.765+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 msgr2=0x55a27fcd0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.765+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00057c0 tx=0x7fa8a0009680 comp rx=0 tx=0).stop 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 shutdown_connections 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcc8080 0x55a27fccfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fce27a0 0x55a27fce4be0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcd2940 0x55a27fcd0160 secure :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00057c0 tx=0x7fa8a0009680 comp rx=0 tx=0).stop 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1226795348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fccf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa88eff5640 0 client.16407 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 >> 172.21.15.28:0/1226795348 conn(0x55a27fcc3cf0 msgr2=0x55a27fcc2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 shutdown_connections 2024-09-19T09:35:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 1 -- 172.21.15.28:0/1226795348 wait complete. 2024-09-19T09:35:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.766+0000 7fa8b369a740 20 client.16407 trim_cache size 0 max 16384 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.783+0000 7fa8b369a740 1 Processor -- start 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.783+0000 7fa8b369a740 1 -- start start 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1af6d0 con 0x55a27fcc8080 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1b12d0 con 0x55a27fcea4b0 2024-09-19T09:35:57.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8b369a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1afda0 con 0x55a27fcc7760 2024-09-19T09:35:57.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55286/0 (socket says 172.21.15.28:55286) 2024-09-19T09:35:57.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1844319261 learned_addr learned my addr 172.21.15.28:0/1844319261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:57.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.784+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1844319261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 msgr2=0x55a27fcca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1844319261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 msgr2=0x55a27fcea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 -- 172.21.15.28:0/1844319261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27fcc3b20 con 0x55a27fcc8080 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009bf0 tx=0x7fa89c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceee4be5c1d62f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa890ff9640 1 -- 172.21.15.28:0/1844319261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c01e040 con 0x55a27fcc8080 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa890ff9640 1 -- 172.21.15.28:0/1844319261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c0053a0 con 0x55a27fcc8080 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.785+0000 7fa890ff9640 1 -- 172.21.15.28:0/1844319261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c026670 con 0x55a27fcc8080 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 -- 172.21.15.28:0/1844319261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 msgr2=0x55a27fcd15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009bf0 tx=0x7fa89c009f60 comp rx=0 tx=0).stop 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 -- 172.21.15.28:0/1844319261 shutdown_connections 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc8080 0x55a27fcd15b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 --2- 172.21.15.28:0/1844319261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc7760 0x55a27fcca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.786+0000 7fa8b369a740 1 -- 172.21.15.28:0/1844319261 >> 172.21.15.28:0/1844319261 conn(0x55a27fcc3cf0 msgr2=0x55a27fcc2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:35:57.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.787+0000 7fa8b369a740 1 -- 172.21.15.28:0/1844319261 shutdown_connections 2024-09-19T09:35:57.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.787+0000 7fa8b369a740 1 -- 172.21.15.28:0/1844319261 wait complete. 2024-09-19T09:35:57.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 -- start start 2024-09-19T09:35:57.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 Processor -- start 2024-09-19T09:35:57.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc7760 0x55a27fcdad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1b0b40 con 0x55a27fcc7760 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1b0470 con 0x55a27fcea4b0 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ad6b0 con 0x55a27fcc8080 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.788+0000 7fa8b369a740 10 client.0 _ll_register_callbacks cb 0x7fa8a00317c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aa7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc7760 0x55a27fcdad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35612/0 (socket says 172.21.15.28:35612) 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 -- 172.21.15.28:0/4090481005 learned_addr learned my addr 172.21.15.28:0/4090481005 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 -- 172.21.15.28:0/4090481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 msgr2=0x55a27fcd72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcd72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 -- 172.21.15.28:0/4090481005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc7760 msgr2=0x55a27fcdad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc7760 0x55a27fcdad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 -- 172.21.15.28:0/4090481005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a27fcd7b20 con 0x55a27fcc8080 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcd72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa8aaffd640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01c040 tx=0x7fa89c009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d50038e6af35269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.789+0000 7fa88eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c01e050 con 0x55a27fcc8080 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fa89c005550 con 0x55a27fcc8080 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c0053a0 con 0x55a27fcc8080 2024-09-19T09:35:57.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa89c042070 con 0x55a27fcc8080 2024-09-19T09:35:57.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 10 client.25960 fetch_fsmap learned FSMap version 33 2024-09-19T09:35:57.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a27fd4e0d0 con 0x55a27fcc8080 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c0053a0 con 0x55a27fcc8080 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fa89c03b860 con 0x55a27fcc8080 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 10 client.25960 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 10 client.25960 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 20 client.25960 populate_metadata read hostname 'smithi028' 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fd4e690 0x55a27fd50ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 4 client.25960 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.790+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55a27fc34d20 con 0x55a27fd4e690 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.791+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fd4e690 0x55a27fd50ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.791+0000 7fa8aa7fc640 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fd4e690 0x55a27fd50ab0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480a6a07a745cd29 server_cookie=62f01ce3f832b3a3 in_seq=0 out_seq=0 2024-09-19T09:35:57.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:35:57.791+0000 7fa88eff5640 10 client.25960 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:35:58.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:35:58 smithi078 ceph-mon[35619]: pgmap v437: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 605 op/s 2024-09-19T09:35:58.883 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:35:58 smithi028 ceph-mon[30972]: pgmap v437: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 605 op/s 2024-09-19T09:35:58.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:35:58 smithi038 ceph-mon[35271]: pgmap v437: 97 pgs: 97 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 605 op/s 2024-09-19T09:35:59.039 INFO:tasks.workunit.client.0.smithi028.stdout: 83 1203783 350 798624 337 589890 1089 2024-09-19T09:36:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:00 smithi078 ceph-mon[35619]: pgmap v438: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.20k op/s 2024-09-19T09:36:00.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:00 smithi038 ceph-mon[35271]: pgmap v438: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.20k op/s 2024-09-19T09:36:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:00 smithi028 ceph-mon[30972]: pgmap v438: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.20k op/s 2024-09-19T09:36:02.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:02 smithi078 ceph-mon[35619]: pgmap v439: 97 pgs: 97 active+clean; 266 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 1.25k op/s 2024-09-19T09:36:02.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:02 smithi038 ceph-mon[35271]: pgmap v439: 97 pgs: 97 active+clean; 266 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 1.25k op/s 2024-09-19T09:36:03.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:02 smithi028 ceph-mon[30972]: pgmap v439: 97 pgs: 97 active+clean; 266 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 1.25k op/s 2024-09-19T09:36:04.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:04 smithi078 ceph-mon[35619]: pgmap v440: 97 pgs: 97 active+clean; 276 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-09-19T09:36:04.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:04 smithi038 ceph-mon[35271]: pgmap v440: 97 pgs: 97 active+clean; 276 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-09-19T09:36:05.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:04 smithi028 ceph-mon[30972]: pgmap v440: 97 pgs: 97 active+clean; 276 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-09-19T09:36:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:06 smithi078 ceph-mon[35619]: pgmap v441: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.38k op/s 2024-09-19T09:36:06.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:06 smithi038 ceph-mon[35271]: pgmap v441: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.38k op/s 2024-09-19T09:36:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:06 smithi028 ceph-mon[30972]: pgmap v441: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.38k op/s 2024-09-19T09:36:08.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:08 smithi028 ceph-mon[30972]: pgmap v442: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-09-19T09:36:08.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:08 smithi078 ceph-mon[35619]: pgmap v442: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-09-19T09:36:08.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:08 smithi038 ceph-mon[35271]: pgmap v442: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-09-19T09:36:09.039 INFO:tasks.workunit.client.0.smithi028.stdout: 83 11118 5 9063 7 3593 10 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- 172.21.15.28:0/3232020742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 msgr2=0x7f5068109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 --2- 172.21.15.28:0/3232020742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f5068109cc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5060009ca0 tx=0x7f506002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- 172.21.15.28:0/3232020742 shutdown_connections 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 --2- 172.21.15.28:0/3232020742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f5068109cc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 --2- 172.21.15.28:0/3232020742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5068104f20 0x7f5068107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 --2- 172.21.15.28:0/3232020742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50681025f0 0x7f50681049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- 172.21.15.28:0/3232020742 >> 172.21.15.28:0/3232020742 conn(0x7f50680fc1f0 msgr2=0x7f50680fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- 172.21.15.28:0/3232020742 shutdown_connections 2024-09-19T09:36:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- 172.21.15.28:0/3232020742 wait complete. 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 Processor -- start 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.392+0000 7f5070349640 1 -- start start 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5068104f20 0x7f5068198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f506819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f506810d830 con 0x7f5068107850 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f506810d6b0 con 0x7f5068104f20 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f5070349640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f506810d9b0 con 0x7f50681025f0 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:42210/0 (socket says 172.21.15.28:42210) 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f506819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 -- 172.21.15.28:0/525701126 learned_addr learned my addr 172.21.15.28:0/525701126 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506d8e4640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5068104f20 0x7f5068198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 -- 172.21.15.28:0/525701126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5068104f20 msgr2=0x7f5068198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5068104f20 0x7f5068198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 -- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 msgr2=0x7f506819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f506819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 -- 172.21.15.28:0/525701126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506819fb40 con 0x7f50681025f0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e8e6640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f506819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.393+0000 7f506e0e5640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9f0 tx=0x7f506400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.394+0000 7f505f7fe640 1 -- 172.21.15.28:0/525701126 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506400cd80 con 0x7f50681025f0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.394+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506819fe30 con 0x7f50681025f0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.394+0000 7f505f7fe640 1 -- 172.21.15.28:0/525701126 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5064010910 con 0x7f50681025f0 2024-09-19T09:36:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.394+0000 7f505f7fe640 1 -- 172.21.15.28:0/525701126 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640188f0 con 0x7f50681025f0 2024-09-19T09:36:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.394+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50681a02a0 con 0x7f50681025f0 2024-09-19T09:36:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.396+0000 7f505f7fe640 1 -- 172.21.15.28:0/525701126 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5064010ac0 con 0x7f50681025f0 2024-09-19T09:36:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.396+0000 7f50731bc740 1 Processor -- start 2024-09-19T09:36:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 -- start start 2024-09-19T09:36:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac1eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac21830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56422ab7d2f0 con 0x56422ac199a0 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56422ab25270 con 0x56422ac1f370 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f50731bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56422ab792b0 con 0x56422ac1a2e0 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac1eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45590/0 (socket says 172.21.15.28:45590) 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 -- 172.21.15.28:0/379274753 learned_addr learned my addr 172.21.15.28:0/379274753 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f505f7fe640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 0x7f50540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506d8e4640 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac21830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 0x7f50540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f505f7fe640 1 -- 172.21.15.28:0/525701126 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f50640cfd80 con 0x7f50681025f0 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 -- 172.21.15.28:0/379274753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 msgr2=0x56422ac1eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.397+0000 7f506e8e6640 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac1eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f506e8e6640 1 -- 172.21.15.28:0/379274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 msgr2=0x56422ac21830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f506e8e6640 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac21830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f506e8e6640 1 -- 172.21.15.28:0/379274753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422ab24340 con 0x56422ac199a0 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f506d8e4640 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac21830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f506e8e6640 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5060009430 tx=0x7f5060031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181b41f26aa081fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f505cbfe640 1 -- 172.21.15.28:0/379274753 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060004b40 con 0x56422ac199a0 2024-09-19T09:36:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f505cbfe640 1 -- 172.21.15.28:0/379274753 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5060050070 con 0x56422ac199a0 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.398+0000 7f505cbfe640 1 -- 172.21.15.28:0/379274753 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060005050 con 0x56422ac199a0 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- 172.21.15.28:0/379274753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 msgr2=0x56422ac19da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f506e8e6640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 0x7f50540a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f506004f000 tx=0x7f506004d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5060009430 tx=0x7f5060031db0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- 172.21.15.28:0/379274753 shutdown_connections 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac21830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac1eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 --2- 172.21.15.28:0/379274753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac19da0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- 172.21.15.28:0/379274753 >> 172.21.15.28:0/379274753 conn(0x56422ac15010 msgr2=0x56422ac15410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- 172.21.15.28:0/379274753 shutdown_connections 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- 172.21.15.28:0/379274753 wait complete. 2024-09-19T09:36:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 -- start start 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.399+0000 7f50731bc740 1 Processor -- start 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac2b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56422ac221c0 con 0x56422ac199a0 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56422ab06290 con 0x56422ac1f370 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56422ac22340 con 0x56422ac1a2e0 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 10 client.0 _ll_register_callbacks cb 0x56422ab37140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45594/0 (socket says 172.21.15.28:45594) 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 -- 172.21.15.28:0/3022314190 learned_addr learned my addr 172.21.15.28:0/3022314190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506d8e4640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e8e6640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac2b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 -- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 msgr2=0x56422ac23fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac23fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 -- 172.21.15.28:0/3022314190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 msgr2=0x56422ac2b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac2b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 -- 172.21.15.28:0/3022314190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422a890210 con 0x56422ac199a0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e8e6640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac2b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f506e0e5640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f506400c6a0 tx=0x7f50640a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c4889428b69014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f5051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640d1070 con 0x56422ac199a0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5064010630 con 0x56422ac199a0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.400+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5060050070 con 0x56422ac199a0 2024-09-19T09:36:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f50640a0720 con 0x56422ac199a0 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 10 client.16425 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56422a95c620 con 0x56422ac199a0 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5060050070 con 0x56422ac199a0 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f50640d4040 con 0x56422ac199a0 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 10 client.16425 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 20 client.16425 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac2f720 0x56422ac31b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 4 client.16425 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56422ab8aa20 con 0x56422ac2f720 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f506d8e4640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac2f720 0x56422ac31b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f506d8e4640 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac2f720 0x56422ac31b90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968b0bb5ab31ea93 server_cookie=c57029b7f7c89233 in_seq=0 out_seq=0 2024-09-19T09:36:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.401+0000 7f5051ffb640 10 client.16425 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f5051ffb640 1 -- 172.21.15.28:0/3022314190 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56422ab8aa20 con 0x56422ac2f720 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f5051ffb640 10 client.16425 handle_command_reply: tid=0 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 client.16425 shutdown 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 msgr2=0x56422ac23aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f506400c6a0 tx=0x7f50640a4fe0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 shutdown_connections 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1f370 0x56422ac2b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac2f720 0x56422ac31b90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac23fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 --2- 172.21.15.28:0/3022314190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac199a0 0x56422ac23aa0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f5051ffb640 0 client.16425 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 >> 172.21.15.28:0/3022314190 conn(0x56422ac15010 msgr2=0x56422ac16820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 shutdown_connections 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.404+0000 7f50731bc740 1 -- 172.21.15.28:0/3022314190 wait complete. 2024-09-19T09:36:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.405+0000 7f50731bc740 20 client.16425 trim_cache size 0 max 16384 2024-09-19T09:36:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.419+0000 7f50731bc740 1 Processor -- start 2024-09-19T09:36:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.419+0000 7f50731bc740 1 -- start start 2024-09-19T09:36:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac199a0 0x56422ac15ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1a2e0 0x56422ac2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50681a9010 con 0x56422ac1f370 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f50681aac10 con 0x56422ac1a2e0 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f50731bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50681a96e0 con 0x56422ac199a0 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45600/0 (socket says 172.21.15.28:45600) 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 -- 172.21.15.28:0/2143000415 learned_addr learned my addr 172.21.15.28:0/2143000415 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e0e5640 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac199a0 0x56422ac15ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506d8e4640 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1a2e0 0x56422ac2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 -- 172.21.15.28:0/2143000415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac199a0 msgr2=0x56422ac15ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac199a0 0x56422ac15ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 -- 172.21.15.28:0/2143000415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1a2e0 msgr2=0x56422ac2afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1a2e0 0x56422ac2afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 -- 172.21.15.28:0/2143000415 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422ac32210 con 0x56422ac1f370 2024-09-19T09:36:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f506e8e6640 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5060009a40 tx=0x7f506002f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06db4b92334b13b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.420+0000 7f5053fff640 1 -- 172.21.15.28:0/2143000415 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060005050 con 0x56422ac1f370 2024-09-19T09:36:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f5053fff640 1 -- 172.21.15.28:0/2143000415 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506005c070 con 0x56422ac1f370 2024-09-19T09:36:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f5053fff640 1 -- 172.21.15.28:0/2143000415 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50600055d0 con 0x56422ac1f370 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- 172.21.15.28:0/2143000415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 msgr2=0x56422ac16400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5060009a40 tx=0x7f506002f490 comp rx=0 tx=0).stop 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- 172.21.15.28:0/2143000415 shutdown_connections 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac1a2e0 0x56422ac2afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac16400 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 --2- 172.21.15.28:0/2143000415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac199a0 0x56422ac15ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- 172.21.15.28:0/2143000415 >> 172.21.15.28:0/2143000415 conn(0x56422ac15010 msgr2=0x56422ac23850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- 172.21.15.28:0/2143000415 shutdown_connections 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- 172.21.15.28:0/2143000415 wait complete. 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.421+0000 7f50731bc740 1 -- start start 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 Processor -- start 2024-09-19T09:36:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac199a0 0x56422ac28d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac29270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50681aa480 con 0x56422ac1f370 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f50681a9db0 con 0x56422ac199a0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50681a6ff0 con 0x56422ac1a2e0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f50731bc740 10 client.0 _ll_register_callbacks cb 0x7f506400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45612/0 (socket says 172.21.15.28:45612) 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 -- 172.21.15.28:0/991472607 learned_addr learned my addr 172.21.15.28:0/991472607 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506d8e4640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e0e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac199a0 0x56422ac28d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 -- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 msgr2=0x56422ac29270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac29270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 -- 172.21.15.28:0/991472607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac199a0 msgr2=0x56422ac28d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac199a0 0x56422ac28d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 -- 172.21.15.28:0/991472607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422ac2a820 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f506e8e6640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f506002ade0 tx=0x7f5060030bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d473fca4c4bbba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.422+0000 7f5051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060064020 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f5060005580 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f506005c070 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5060005050 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 10 client.16431 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56422ac9a970 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506005c070 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f50600605e0 con 0x56422ac1f370 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 10 client.16431 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 20 client.16431 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac9b0e0 0x56422ac9d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 4 client.16431 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56422ab84d70 con 0x56422ac9b0e0 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.423+0000 7f506e0e5640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac9b0e0 0x56422ac9d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.424+0000 7f506e0e5640 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac9b0e0 0x56422ac9d500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bad2358192cdf7 server_cookie=4368cd7a9d7260a7 in_seq=0 out_seq=0 2024-09-19T09:36:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:10.424+0000 7f5051ffb640 10 client.16431 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:10.854 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:10 smithi028 ceph-mon[30972]: pgmap v443: 97 pgs: 97 active+clean; 291 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 1.07k op/s 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa88eff5640 1 -- 172.21.15.28:0/4090481005 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a27fc34d20 con 0x55a27fd4e690 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa88eff5640 10 client.25960 handle_command_reply: tid=0 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 client.25960 shutdown 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 msgr2=0x55a27fcdb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01c040 tx=0x7fa89c009970 comp rx=0 tx=0).stop 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 shutdown_connections 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a27fcea4b0 0x55a27fcd72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55a27fd4e690 0x55a27fd50ab0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a27fcc8080 0x55a27fcdb2a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 --2- 172.21.15.28:0/4090481005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a27fcc7760 0x55a27fcdad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa88eff5640 0 client.25960 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:10.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 >> 172.21.15.28:0/4090481005 conn(0x55a27fcc3cf0 msgr2=0x55a27fcc2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 shutdown_connections 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 1 -- 172.21.15.28:0/4090481005 wait complete. 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.853+0000 7fa8b369a740 20 client.25960 trim_cache size 0 max 16384 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 33, 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 14, 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "f6325b98": { 2024-09-19T09:36:10.856 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 14, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 136.6, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:10.857 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 msgr2=0x7fa8940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 0x7fa8940a5780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8a409c250 tx=0x7fa8a4066f90 comp rx=0 tx=0).stop 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 msgr2=0x7fa8ac19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:10.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 0x7fa8ac19cd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009fd0 tx=0x7fa8a400ef10 comp rx=0 tx=0).stop 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 shutdown_connections 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa8940a32c0 0x7fa8940a5780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac106f20 0x7fa8ac19cd50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8ac1027a0 0x7fa8ac1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 --2- 172.21.15.28:0/2090737232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8ac101df0 0x7fa8ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 >> 172.21.15.28:0/2090737232 conn(0x7fa8ac0fd8f0 msgr2=0x7fa8ac0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 shutdown_connections 2024-09-19T09:36:10.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:10.854+0000 7fa8b12c1640 1 -- 172.21.15.28:0/2090737232 wait complete. 2024-09-19T09:36:10.859 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:10 smithi078 ceph-mon[35619]: pgmap v443: 97 pgs: 97 active+clean; 291 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 1.07k op/s 2024-09-19T09:36:10.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:10 smithi038 ceph-mon[35271]: pgmap v443: 97 pgs: 97 active+clean; 291 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 1.07k op/s 2024-09-19T09:36:11.678 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f6325b98', quiesce took 18.2 seconds. Will release after: -1.5999999999999979 2024-09-19T09:36:11.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:36:11.981 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:36:12.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.436+0000 7fcad3387640 1 -- 172.21.15.28:0/1330177290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 msgr2=0x7fcacc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:12.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.436+0000 7fcad3387640 1 --2- 172.21.15.28:0/1330177290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc104e80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac002a410 comp rx=0 tx=0).stop 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- 172.21.15.28:0/1330177290 shutdown_connections 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- 172.21.15.28:0/1330177290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 0x7fcacc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- 172.21.15.28:0/1330177290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc104e80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- 172.21.15.28:0/1330177290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 0x7fcacc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- 172.21.15.28:0/1330177290 >> 172.21.15.28:0/1330177290 conn(0x7fcacc077210 msgr2=0x7fcacc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- 172.21.15.28:0/1330177290 shutdown_connections 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- 172.21.15.28:0/1330177290 wait complete. 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 Processor -- start 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- start start 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 0x7fcacc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 0x7fcacc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:12.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10fd90 con 0x7fcacc100e30 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10fc10 con 0x7fcacc100480 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.437+0000 7fcad3387640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10ff10 con 0x7fcacc1055b0 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45630/0 (socket says 172.21.15.28:45630) 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 -- 172.21.15.28:0/2691514736 learned_addr learned my addr 172.21.15.28:0/2691514736 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad1924640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 0x7fcacc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad1123640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 0x7fcacc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 -- 172.21.15.28:0/2691514736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 msgr2=0x7fcacc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 0x7fcacc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 -- 172.21.15.28:0/2691514736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 msgr2=0x7fcacc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:12.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 0x7fcacc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 -- 172.21.15.28:0/2691514736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc19d440 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad0922640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac003f030 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc19d6d0 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0003ce0 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0035070 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.438+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a5010 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.439+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc100860 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.441+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcac0042030 con 0x7fcacc100e30 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.442+0000 7fcabe7fc640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 0x7fcab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:12.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.442+0000 7fcad1123640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 0x7fcab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:12.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.442+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7fcac00effe0 con 0x7fcacc100e30 2024-09-19T09:36:12.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.442+0000 7fcad1123640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 0x7fcab80a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bbe0 tx=0x7fcac800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:12.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.445+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcac00bc7a0 con 0x7fcacc100e30 2024-09-19T09:36:12.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.658+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcacc1997f0 con 0x7fcacc100e30 2024-09-19T09:36:12.660 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:12 smithi028 ceph-mon[30972]: pgmap v444: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 436 op/s 2024-09-19T09:36:12.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.660+0000 7fcabe7fc640 1 -- 172.21.15.28:0/2691514736 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11957 (secure 0 0 0) 0x7fcacc1997f0 con 0x7fcacc100e30 2024-09-19T09:36:12.661 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:36:12.661 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"btime":"2024-09-19T09:35:21:300725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":25804,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:35:21.216576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:36:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 msgr2=0x7fcab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 0x7fcab80a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bbe0 tx=0x7fcac800b390 comp rx=0 tx=0).stop 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 msgr2=0x7fcacc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac00042f0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 shutdown_connections 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcab80a3240 0x7fcab80a5700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc1055b0 0x7fcacc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcacc100e30 0x7fcacc1989d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 --2- 172.21.15.28:0/2691514736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcacc100480 0x7fcacc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 >> 172.21.15.28:0/2691514736 conn(0x7fcacc077210 msgr2=0x7fcacc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 shutdown_connections 2024-09-19T09:36:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:12.662+0000 7fcad3387640 1 -- 172.21.15.28:0/2691514736 wait complete. 2024-09-19T09:36:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:12 smithi078 ceph-mon[35619]: pgmap v444: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 436 op/s 2024-09-19T09:36:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:12 smithi038 ceph-mon[35271]: pgmap v444: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 436 op/s 2024-09-19T09:36:13.598 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id f6325b98 --release --await' 2024-09-19T09:36:13.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id f6325b98 --release --await 2024-09-19T09:36:13.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2691514736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:13.861 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:36:13.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2691514736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:13.979 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2691514736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:14.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 -- 172.21.15.28:0/214114422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec102ef0 msgr2=0x7f4aec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 --2- 172.21.15.28:0/214114422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec102ef0 0x7f4aec10da40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009ca0 tx=0x7f4ae402a950 comp rx=0 tx=0).stop 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 -- 172.21.15.28:0/214114422 shutdown_connections 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 --2- 172.21.15.28:0/214114422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec102ef0 0x7f4aec10da40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 --2- 172.21.15.28:0/214114422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 0x7f4aec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 --2- 172.21.15.28:0/214114422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec100e20 0x7f4aec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 -- 172.21.15.28:0/214114422 >> 172.21.15.28:0/214114422 conn(0x7f4aec0fc7d0 msgr2=0x7f4aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 -- 172.21.15.28:0/214114422 shutdown_connections 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.315+0000 7f4af2704640 1 -- 172.21.15.28:0/214114422 wait complete. 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 Processor -- start 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 -- start start 2024-09-19T09:36:14.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 0x7f4aec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec102ef0 0x7f4aec19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1120a0 con 0x7f4aec100e20 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec111f20 con 0x7f4aec102ef0 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.316+0000 7f4af2704640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec112220 con 0x7f4aec102570 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38712/0 (socket says 172.21.15.28:38712) 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 -- 172.21.15.28:0/4292914569 learned_addr learned my addr 172.21.15.28:0/4292914569 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 0x7f4aec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec102ef0 0x7f4aec19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 -- 172.21.15.28:0/4292914569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 msgr2=0x7f4aec198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 0x7f4aec198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 -- 172.21.15.28:0/4292914569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec102ef0 msgr2=0x7f4aec19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec102ef0 0x7f4aec19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 -- 172.21.15.28:0/4292914569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec19d4b0 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00e9f0 tx=0x7f4adc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4ae9ffb640 1 -- 172.21.15.28:0/4292914569 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00cd80 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4ae9ffb640 1 -- 172.21.15.28:0/4292914569 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4adc005500 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec19d7a0 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4ae9ffb640 1 -- 172.21.15.28:0/4292914569 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc010690 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.317+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec1a5080 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4ae9ffb640 1 -- 172.21.15.28:0/4292914569 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4adc0046c0 con 0x7f4aec100e20 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 Processor -- start 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 -- start start 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x5624377f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562437747620 con 0x5624377e2840 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624376eee50 con 0x5624377f1650 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af4add740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624377435e0 con 0x5624377e1f20 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:58342/0 (socket says 172.21.15.28:58342) 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4af0ca1640 1 -- 172.21.15.28:0/1088837686 learned_addr learned my addr 172.21.15.28:0/1088837686 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.320+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x5624377f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1088837686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 msgr2=0x5624377e2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1088837686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 msgr2=0x5624377f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1088837686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624376dfd40 con 0x5624377f1650 2024-09-19T09:36:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae9ffb640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 0x7f4ad80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 0x7f4ad80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x5624377f3a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009bd0 tx=0x7f4ad4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c8f111c7744e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae9ffb640 1 -- 172.21.15.28:0/4292914569 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4adc0cfc60 con 0x7f4aec100e20 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae27fc640 1 -- 172.21.15.28:0/1088837686 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004050 con 0x5624377f1650 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae27fc640 1 -- 172.21.15.28:0/1088837686 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4005500 con 0x5624377f1650 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.321+0000 7f4ae27fc640 1 -- 172.21.15.28:0/1088837686 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004d40 con 0x5624377f1650 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- 172.21.15.28:0/1088837686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 msgr2=0x5624377f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x5624377f3a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009bd0 tx=0x7f4ad4011810 comp rx=0 tx=0).stop 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 0x7f4ad80a5780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009ca0 tx=0x7f4ae4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- 172.21.15.28:0/1088837686 shutdown_connections 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x5624377f3a40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 --2- 172.21.15.28:0/1088837686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377e2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- 172.21.15.28:0/1088837686 >> 172.21.15.28:0/1088837686 conn(0x5624377de5f0 msgr2=0x5624377de9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- 172.21.15.28:0/1088837686 shutdown_connections 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- 172.21.15.28:0/1088837686 wait complete. 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.322+0000 7f4af4add740 1 -- start start 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 Processor -- start 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 0x5624377ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5624377f4220 con 0x5624377f1650 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5624376cff10 con 0x5624377e1f20 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624377f43a0 con 0x5624377e2840 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af4add740 10 client.0 _ll_register_callbacks cb 0x5624376f3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 0x5624377ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:58358/0 (socket says 172.21.15.28:58358) 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38724/0 (socket says 172.21.15.28:38724) 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1077473380 learned_addr learned my addr 172.21.15.28:0/1077473380 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1077473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 msgr2=0x5624377ee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 0x5624377ee560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1077473380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 msgr2=0x5624377eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4ae3fff640 1 -- 172.21.15.28:0/1077473380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562437456580 con 0x5624377e2840 2024-09-19T09:36:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4aebfff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 0x5624377ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.323+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ad401b040 tx=0x7f4ad40038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e475d8c7d8027ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4029070 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad4005500 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4ad4003ca0 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4ad4035050 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4af4add740 10 client.25996 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562437520960 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4005500 con 0x5624377e2840 2024-09-19T09:36:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f4ad4023430 con 0x5624377e2840 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.324+0000 7f4af4add740 10 client.25996 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4af4add740 10 client.25996 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4af4add740 20 client.25996 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5624378010c0 0x5624378034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4af4add740 4 client.25996 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562437754b90 con 0x5624378010c0 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4aebfff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5624378010c0 0x5624378034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4aebfff640 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5624378010c0 0x5624378034b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6212adb168497501 server_cookie=d47aebecd245e3f3 in_seq=0 out_seq=0 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.325+0000 7f4acb7fe640 10 client.25996 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4acb7fe640 1 -- 172.21.15.28:0/1077473380 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562437754b90 con 0x5624378010c0 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4acb7fe640 10 client.25996 handle_command_reply: tid=0 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 client.25996 shutdown 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 msgr2=0x5624377eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ad401b040 tx=0x7f4ad40038a0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 shutdown_connections 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5624378010c0 0x5624378034b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e2840 0x5624377eeaa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 --2- 172.21.15.28:0/1077473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e1f20 0x5624377ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4acb7fe640 0 client.25996 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 >> 172.21.15.28:0/1077473380 conn(0x5624377de5f0 msgr2=0x5624377e4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 shutdown_connections 2024-09-19T09:36:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.328+0000 7f4af4add740 1 -- 172.21.15.28:0/1077473380 wait complete. 2024-09-19T09:36:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.329+0000 7f4af4add740 20 client.25996 trim_cache size 0 max 16384 2024-09-19T09:36:14.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 Processor -- start 2024-09-19T09:36:14.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 -- start start 2024-09-19T09:36:14.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x562437808d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e2840 0x562437809240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ab270 con 0x5624377f1650 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ace70 con 0x5624377e2840 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af4add740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ab940 con 0x5624377e1f20 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38730/0 (socket says 172.21.15.28:38730) 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x562437808d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 -- 172.21.15.28:0/1971045412 learned_addr learned my addr 172.21.15.28:0/1971045412 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e2840 0x562437809240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 -- 172.21.15.28:0/1971045412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 msgr2=0x562437808d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x562437808d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 -- 172.21.15.28:0/1971045412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e2840 msgr2=0x562437809240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e2840 0x562437809240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.347+0000 7f4af0ca1640 1 -- 172.21.15.28:0/1971045412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624377e2670 con 0x5624377f1650 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.348+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4ae402f4e0 tx=0x7f4ae4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ee7e574dc2f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.348+0000 7f4ae1ffb640 1 -- 172.21.15.28:0/1971045412 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4056070 con 0x5624377f1650 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.348+0000 7f4ae1ffb640 1 -- 172.21.15.28:0/1971045412 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae4004410 con 0x5624377f1650 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.348+0000 7f4ae1ffb640 1 -- 172.21.15.28:0/1971045412 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4034040 con 0x5624377f1650 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- 172.21.15.28:0/1971045412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 msgr2=0x5624377eb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4ae402f4e0 tx=0x7f4ae4045000 comp rx=0 tx=0).stop 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- 172.21.15.28:0/1971045412 shutdown_connections 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377e2840 0x562437809240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377f1650 0x5624377eb7c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 --2- 172.21.15.28:0/1971045412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x562437808d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- 172.21.15.28:0/1971045412 >> 172.21.15.28:0/1971045412 conn(0x5624377de5f0 msgr2=0x5624377ece30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- 172.21.15.28:0/1971045412 shutdown_connections 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- 172.21.15.28:0/1971045412 wait complete. 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 -- start start 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.349+0000 7f4af4add740 1 Processor -- start 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x56243786acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ac6e0 con 0x5624377e2840 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ac010 con 0x5624377f1650 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1a9270 con 0x5624377e1f20 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 10 client.0 _ll_register_callbacks cb 0x7f4ad4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38740/0 (socket says 172.21.15.28:38740) 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:58384/0 (socket says 172.21.15.28:58384) 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 -- 172.21.15.28:0/4238407603 learned_addr learned my addr 172.21.15.28:0/4238407603 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x56243786acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 -- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 msgr2=0x5624377f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 -- 172.21.15.28:0/4238407603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 msgr2=0x56243786acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x56243786acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 -- 172.21.15.28:0/4238407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624377f6530 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af0ca1640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x56243786acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4ae3fff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009b20 tx=0x7f4ad4003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ede350c8178621 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4acb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4029040 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f4ad4004680 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.350+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae4004410 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4ad4023430 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 10 client.16455 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56243786c360 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae4004410 con 0x5624377e2840 2024-09-19T09:36:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f4ad402e560 con 0x5624377e2840 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 10 client.16455 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 10 client.16455 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 20 client.16455 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56243786ca80 0x56243786eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 4 client.16455 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f6325b98", "release": true, "await": true}] 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f6325b98", "release": true, "await": true}) -- 0x56243774f0a0 con 0x56243786ca80 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.351+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56243786ca80 0x56243786eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.352+0000 7f4aebfff640 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56243786ca80 0x56243786eea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c8cad513d9000a server_cookie=7d43afbfd881d63c in_seq=0 out_seq=0 2024-09-19T09:36:14.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.352+0000 7f4acb7fe640 10 client.16455 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:14 smithi078 ceph-mon[35619]: pgmap v445: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-19T09:36:14.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:14 smithi038 ceph-mon[35271]: pgmap v445: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-19T09:36:14.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.985+0000 7f4acb7fe640 1 -- 172.21.15.28:0/4238407603 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56243774f0a0 con 0x56243786ca80 2024-09-19T09:36:14.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4acb7fe640 10 client.16455 handle_command_reply: tid=0 2024-09-19T09:36:14.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 client.16455 shutdown 2024-09-19T09:36:14.987 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:14 smithi028 ceph-mon[30972]: pgmap v445: 97 pgs: 97 active+clean; 295 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-09-19T09:36:14.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 msgr2=0x5624377f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009b20 tx=0x7f4ad4003980 comp rx=0 tx=0).stop 2024-09-19T09:36:14.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 shutdown_connections 2024-09-19T09:36:14.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5624377f1650 0x56243786acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56243786ca80 0x56243786eea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5624377e2840 0x5624377f5c50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 --2- 172.21.15.28:0/4238407603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624377e1f20 0x5624377f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 >> 172.21.15.28:0/4238407603 conn(0x5624377de5f0 msgr2=0x5624377dcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4acb7fe640 0 client.16455 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 shutdown_connections 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.986+0000 7f4af4add740 1 -- 172.21.15.28:0/4238407603 wait complete. 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af4add740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 33, 2024-09-19T09:36:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 16, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "f6325b98": { 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 16, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 136.6, 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:36:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:36:14.991 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 msgr2=0x7f4ad80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 0x7f4ad80a5780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009ca0 tx=0x7f4ae4005c50 comp rx=0 tx=0).stop 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 msgr2=0x7f4aec198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00e9f0 tx=0x7f4adc00edf0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 shutdown_connections 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f4ad80a32c0 0x7f4ad80a5780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4aec102ef0 0x7f4aec19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4aec102570 0x7f4aec198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 --2- 172.21.15.28:0/4292914569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4aec100e20 0x7f4aec198500 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.987+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 >> 172.21.15.28:0/4292914569 conn(0x7f4aec0fc7d0 msgr2=0x7f4aec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.988+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 shutdown_connections 2024-09-19T09:36:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:14.988+0000 7f4af2704640 1 -- 172.21.15.28:0/4292914569 wait complete. 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f5051ffb640 1 -- 172.21.15.28:0/991472607 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56422ab84d70 con 0x56422ac9b0e0 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f5051ffb640 10 client.16431 handle_command_reply: tid=0 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 client.16431 shutdown 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 msgr2=0x56422ac2a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f506002ade0 tx=0x7f5060030bc0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 shutdown_connections 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56422ac9b0e0 0x56422ac9d500 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56422ac1f370 0x56422ac2a210 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56422ac1a2e0 0x56422ac29270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 --2- 172.21.15.28:0/991472607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56422ac199a0 0x56422ac28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.070+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 >> 172.21.15.28:0/991472607 conn(0x56422ac15010 msgr2=0x56422ac139a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.071+0000 7f5051ffb640 0 client.16431 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.071+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 shutdown_connections 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.071+0000 7f50731bc740 1 -- 172.21.15.28:0/991472607 wait complete. 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.071+0000 7f50731bc740 20 client.16431 trim_cache size 0 max 16384 2024-09-19T09:36:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:36:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:36:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:36:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:36:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.072+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 msgr2=0x7f50540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.072+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 0x7f50540a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f506004f000 tx=0x7f506004d000 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.072+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 msgr2=0x7f5068197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.072+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9f0 tx=0x7f506400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 shutdown_connections 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f50540a3240 0x7f50540a5700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5068104f20 0x7f5068198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5068107850 0x7f506819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 --2- 172.21.15.28:0/525701126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50681025f0 0x7f5068197f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 >> 172.21.15.28:0/525701126 conn(0x7f50680fc1f0 msgr2=0x7f5068102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 shutdown_connections 2024-09-19T09:36:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:15.073+0000 7f5070349640 1 -- 172.21.15.28:0/525701126 wait complete. 2024-09-19T09:36:15.725 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f6325b98', total seconds elapsed: 22.3 2024-09-19T09:36:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:16 smithi078 ceph-mon[35619]: pgmap v446: 97 pgs: 97 active+clean; 302 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 334 op/s 2024-09-19T09:36:16.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:16 smithi038 ceph-mon[35271]: pgmap v446: 97 pgs: 97 active+clean; 302 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 334 op/s 2024-09-19T09:36:17.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:16 smithi028 ceph-mon[30972]: pgmap v446: 97 pgs: 97 active+clean; 302 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 334 op/s 2024-09-19T09:36:18.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:18 smithi078 ceph-mon[35619]: pgmap v447: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 410 op/s 2024-09-19T09:36:18.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:18 smithi038 ceph-mon[35271]: pgmap v447: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 410 op/s 2024-09-19T09:36:19.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:18 smithi028 ceph-mon[30972]: pgmap v447: 97 pgs: 97 active+clean; 286 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 410 op/s 2024-09-19T09:36:19.039 INFO:tasks.workunit.client.0.smithi028.stdout: 86 5351 53 4498 75 2694 104 2024-09-19T09:36:20.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:20 smithi078 ceph-mon[35619]: pgmap v448: 97 pgs: 97 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-09-19T09:36:20.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:20 smithi038 ceph-mon[35271]: pgmap v448: 97 pgs: 97 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-09-19T09:36:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:20 smithi028 ceph-mon[30972]: pgmap v448: 97 pgs: 97 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-09-19T09:36:22.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:22 smithi078 ceph-mon[35619]: pgmap v449: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-19T09:36:22.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:22 smithi038 ceph-mon[35271]: pgmap v449: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-19T09:36:23.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:22 smithi028 ceph-mon[30972]: pgmap v449: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-19T09:36:24.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:24 smithi078 ceph-mon[35619]: pgmap v450: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-19T09:36:24.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:24 smithi038 ceph-mon[35271]: pgmap v450: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-19T09:36:25.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:24 smithi028 ceph-mon[30972]: pgmap v450: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-19T09:36:26.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:26 smithi078 ceph-mon[35619]: pgmap v451: 97 pgs: 97 active+clean; 281 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-09-19T09:36:26.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:26 smithi038 ceph-mon[35271]: pgmap v451: 97 pgs: 97 active+clean; 281 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-09-19T09:36:27.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:26 smithi028 ceph-mon[30972]: pgmap v451: 97 pgs: 97 active+clean; 281 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.172+0000 7f94cf3c4640 1 -- 172.21.15.28:0/3263970308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c8101c30 msgr2=0x7f94c81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.172+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/3263970308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c8101c30 0x7f94c81020b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009900 tx=0x7f94b802a410 comp rx=0 tx=0).stop 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 -- 172.21.15.28:0/3263970308 shutdown_connections 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/3263970308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c81025f0 0x7f94c8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/3263970308 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c8101c30 0x7f94c81020b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/3263970308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 -- 172.21.15.28:0/3263970308 >> 172.21.15.28:0/3263970308 conn(0x7f94c80fc1c0 msgr2=0x7f94c80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 -- 172.21.15.28:0/3263970308 shutdown_connections 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 -- 172.21.15.28:0/3263970308 wait complete. 2024-09-19T09:36:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 Processor -- start 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.173+0000 7f94cf3c4640 1 -- start start 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 0x7f94c8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810c6f0 con 0x7f94c81025f0 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810c570 con 0x7f94c8101c30 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810c870 con 0x7f94c8100a40 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 0x7f94c8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58534/0 (socket says 172.21.15.28:58534) 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 -- 172.21.15.28:0/261298690 learned_addr learned my addr 172.21.15.28:0/261298690 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cc95f640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 0x7f94c8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:35492/0 (socket says 172.21.15.28:35492) 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 -- 172.21.15.28:0/261298690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 msgr2=0x7f94c8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 -- 172.21.15.28:0/261298690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 msgr2=0x7f94c8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 0x7f94c8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 -- 172.21.15.28:0/261298690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c819fb10 con 0x7f94c81025f0 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd160640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cd961640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f94c000efd0 tx=0x7f94c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94be7fc640 1 -- 172.21.15.28:0/261298690 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0003b70 con 0x7f94c81025f0 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94be7fc640 1 -- 172.21.15.28:0/261298690 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c0003d10 con 0x7f94c81025f0 2024-09-19T09:36:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94be7fc640 1 -- 172.21.15.28:0/261298690 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00109f0 con 0x7f94c81025f0 2024-09-19T09:36:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c819fe00 con 0x7f94c81025f0 2024-09-19T09:36:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.174+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a02c0 con 0x7f94c81025f0 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94be7fc640 1 -- 172.21.15.28:0/261298690 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f94c0004020 con 0x7f94c81025f0 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 Processor -- start 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 -- start start 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d19a0 0x56511c4d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56511c4352f0 con 0x56511c4d7370 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56511c3dd270 con 0x56511c4d22e0 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.177+0000 7f94d223b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56511c4312b0 con 0x56511c4d19a0 2024-09-19T09:36:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:35504/0 (socket says 172.21.15.28:35504) 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 -- 172.21.15.28:0/820469637 learned_addr learned my addr 172.21.15.28:0/820469637 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cc95f640 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d19a0 0x56511c4d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94be7fc640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 0x7f94b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 -- 172.21.15.28:0/820469637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d19a0 msgr2=0x56511c4d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d19a0 0x56511c4d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd961640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 0x7f94b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 -- 172.21.15.28:0/820469637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 msgr2=0x56511c4d9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4d9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 -- 172.21.15.28:0/820469637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511c3dc340 con 0x56511c4d22e0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94be7fc640 1 -- 172.21.15.28:0/261298690 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94c00cfc70 con 0x7f94c81025f0 2024-09-19T09:36:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94cd160640 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94c40114c0 tx=0x7f94c4011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf58f10ed3132ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94b37fe640 1 -- 172.21.15.28:0/820469637 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004020 con 0x56511c4d22e0 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94b37fe640 1 -- 172.21.15.28:0/820469637 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c4005500 con 0x56511c4d22e0 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.178+0000 7f94b37fe640 1 -- 172.21.15.28:0/820469637 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004e10 con 0x56511c4d22e0 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- 172.21.15.28:0/820469637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 msgr2=0x56511c4d6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94c40114c0 tx=0x7f94c4011980 comp rx=0 tx=0).stop 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94cd961640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 0x7f94b40a5780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f94c000aeb0 tx=0x7f94c006b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- 172.21.15.28:0/820469637 shutdown_connections 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4d6da0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 --2- 172.21.15.28:0/820469637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d19a0 0x56511c4d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- 172.21.15.28:0/820469637 >> 172.21.15.28:0/820469637 conn(0x56511c4cd010 msgr2=0x56511c4cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- 172.21.15.28:0/820469637 shutdown_connections 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- 172.21.15.28:0/820469637 wait complete. 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.179+0000 7f94d223b740 1 -- start start 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 Processor -- start 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d7370 0x56511c4e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56511c4da340 con 0x56511c4d19a0 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56511c3be290 con 0x56511c4d7370 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56511c4da4c0 con 0x56511c4d22e0 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94d223b740 10 client.0 _ll_register_callbacks cb 0x56511c3ef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:57642/0 (socket says 172.21.15.28:57642) 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 -- 172.21.15.28:0/3947402326 learned_addr learned my addr 172.21.15.28:0/3947402326 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cc95f640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d7370 0x56511c4e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 -- 172.21.15.28:0/3947402326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d7370 msgr2=0x56511c4e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d7370 0x56511c4e3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 -- 172.21.15.28:0/3947402326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 msgr2=0x56511c4dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.180+0000 7f94cd160640 1 -- 172.21.15.28:0/3947402326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511c148210 con 0x56511c4d22e0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94cd160640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011e80 tx=0x7f94c4005100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1e38cb75cdf57d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4028020 con 0x56511c4d22e0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94c40052a0 con 0x56511c4d22e0 2024-09-19T09:36:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c4005500 con 0x56511c4d22e0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94c401d040 con 0x56511c4d22e0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94d223b740 10 client.26014 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56511c214620 con 0x56511c4d22e0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.181+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c4005500 con 0x56511c4d22e0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f94c4023470 con 0x56511c4d22e0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 10 client.26014 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 20 client.26014 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c4e75c0 0x56511c4e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 4 client.26014 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56511c442a20 con 0x56511c4e75c0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94cc95f640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c4e75c0 0x56511c4e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94b0ff9640 10 client.26014 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.182+0000 7f94cc95f640 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c4e75c0 0x56511c4e9a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ceea5f9dbec3dc7 server_cookie=db26ad96a405e3ef in_seq=0 out_seq=0 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94b0ff9640 1 -- 172.21.15.28:0/3947402326 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56511c442a20 con 0x56511c4e75c0 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94b0ff9640 10 client.26014 handle_command_reply: tid=0 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 client.26014 shutdown 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 msgr2=0x56511c4dc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011e80 tx=0x7f94c4005100 comp rx=0 tx=0).stop 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 shutdown_connections 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d7370 0x56511c4e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c4e75c0 0x56511c4e9a20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4dc160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 --2- 172.21.15.28:0/3947402326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 >> 172.21.15.28:0/3947402326 conn(0x56511c4cd010 msgr2=0x56511c4ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94b0ff9640 0 client.26014 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 shutdown_connections 2024-09-19T09:36:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 1 -- 172.21.15.28:0/3947402326 wait complete. 2024-09-19T09:36:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.183+0000 7f94d223b740 20 client.26014 trim_cache size 0 max 16384 2024-09-19T09:36:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 Processor -- start 2024-09-19T09:36:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 -- start start 2024-09-19T09:36:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d19a0 0x56511c4db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81a8f70 con 0x56511c4d7370 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81aab70 con 0x56511c4d19a0 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94d223b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81a9640 con 0x56511c4d22e0 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58566/0 (socket says 172.21.15.28:58566) 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94cd160640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94cc95f640 1 -- 172.21.15.28:0/2274174551 learned_addr learned my addr 172.21.15.28:0/2274174551 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.198+0000 7f94cc95f640 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d19a0 0x56511c4db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 -- 172.21.15.28:0/2274174551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 msgr2=0x56511c4e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 -- 172.21.15.28:0/2274174551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d19a0 msgr2=0x56511c4db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d19a0 0x56511c4db0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 -- 172.21.15.28:0/2274174551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511c4da220 con 0x56511c4d7370 2024-09-19T09:36:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94cc95f640 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009850 tx=0x7f94b8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6535edc9dbc732 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94b2ffd640 1 -- 172.21.15.28:0/2274174551 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8052070 con 0x56511c4d7370 2024-09-19T09:36:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94b2ffd640 1 -- 172.21.15.28:0/2274174551 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b8053040 con 0x56511c4d7370 2024-09-19T09:36:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.199+0000 7f94b2ffd640 1 -- 172.21.15.28:0/2274174551 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8035040 con 0x56511c4d7370 2024-09-19T09:36:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- 172.21.15.28:0/2274174551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 msgr2=0x56511c4cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009850 tx=0x7f94b8005780 comp rx=0 tx=0).stop 2024-09-19T09:36:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- 172.21.15.28:0/2274174551 shutdown_connections 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d7370 0x56511c4cc0b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d22e0 0x56511c4e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 --2- 172.21.15.28:0/2274174551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d19a0 0x56511c4db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- 172.21.15.28:0/2274174551 >> 172.21.15.28:0/2274174551 conn(0x56511c4cd010 msgr2=0x56511c4cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- 172.21.15.28:0/2274174551 shutdown_connections 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- 172.21.15.28:0/2274174551 wait complete. 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 -- start start 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.200+0000 7f94d223b740 1 Processor -- start 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4e0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d7370 0x56511c4e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81aa3e0 con 0x56511c4d19a0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81a9d10 con 0x56511c4d22e0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81a6f50 con 0x56511c4d7370 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94d223b740 10 client.0 _ll_register_callbacks cb 0x7f94c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cc95f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58578/0 (socket says 172.21.15.28:58578) 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cc95f640 1 -- 172.21.15.28:0/4338336 learned_addr learned my addr 172.21.15.28:0/4338336 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4e0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cc95f640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d7370 0x56511c4e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 -- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d7370 msgr2=0x56511c4e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d7370 0x56511c4e1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 -- 172.21.15.28:0/4338336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 msgr2=0x56511c4e06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4e06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 -- 172.21.15.28:0/4338336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511c4e2240 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94cd160640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4e0c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009380 tx=0x7f94c4023cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26041cf098001950 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.201+0000 7f94b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4028040 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f94c4004640 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b8053040 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94c4004860 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 10 client.26048 fetch_fsmap learned FSMap version 33 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56511c5522c0 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b8053040 con 0x56511c4d22e0 2024-09-19T09:36:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f94c400f770 con 0x56511c4d22e0 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 10 client.26048 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 20 client.26048 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c552ca0 0x56511c5550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 4 client.26048 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.202+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56511c43cd70 con 0x56511c552ca0 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.203+0000 7f94cc95f640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c552ca0 0x56511c5550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.203+0000 7f94b0ff9640 10 client.26048 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:28.203+0000 7f94cc95f640 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c552ca0 0x56511c5550c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c01fe90bdd837b5 server_cookie=408974ab7e70c4df in_seq=0 out_seq=0 2024-09-19T09:36:28.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:28 smithi078 ceph-mon[35619]: pgmap v452: 97 pgs: 97 active+clean; 284 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-09-19T09:36:28.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:28 smithi038 ceph-mon[35271]: pgmap v452: 97 pgs: 97 active+clean; 284 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-09-19T09:36:29.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:28 smithi028 ceph-mon[30972]: pgmap v452: 97 pgs: 97 active+clean; 284 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-09-19T09:36:29.039 INFO:tasks.workunit.client.0.smithi028.stdout: 90 92904 207 61788 162 44806 380 2024-09-19T09:36:30.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:30 smithi078 ceph-mon[35619]: pgmap v453: 97 pgs: 97 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:36:30.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:30 smithi038 ceph-mon[35271]: pgmap v453: 97 pgs: 97 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:36:31.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:30 smithi028 ceph-mon[30972]: pgmap v453: 97 pgs: 97 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:36:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.436+0000 7f94b0ff9640 1 -- 172.21.15.28:0/4338336 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56511c43cd70 con 0x56511c552ca0 2024-09-19T09:36:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.436+0000 7f94b0ff9640 10 client.26048 handle_command_reply: tid=0 2024-09-19T09:36:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.444+0000 7f94d223b740 1 client.26048 shutdown 2024-09-19T09:36:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 msgr2=0x56511c4e0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4e0c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009380 tx=0x7f94c4023cb0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 shutdown_connections 2024-09-19T09:36:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56511c4d22e0 0x56511c4e0c20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56511c552ca0 0x56511c5550c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56511c4d7370 0x56511c4e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 --2- 172.21.15.28:0/4338336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56511c4d19a0 0x56511c4e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 >> 172.21.15.28:0/4338336 conn(0x56511c4cd010 msgr2=0x56511c4dafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.447+0000 7f94b0ff9640 0 client.26048 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.450+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 shutdown_connections 2024-09-19T09:36:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.450+0000 7f94d223b740 1 -- 172.21.15.28:0/4338336 wait complete. 2024-09-19T09:36:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.451+0000 7f94d223b740 20 client.26048 trim_cache size 0 max 16384 2024-09-19T09:36:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:36:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:36:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:36:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:36:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.674+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 msgr2=0x7f94b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.674+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 0x7f94b40a5780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f94c000aeb0 tx=0x7f94c006b000 comp rx=0 tx=0).stop 2024-09-19T09:36:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.674+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 msgr2=0x7f94c819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.674+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f94c000efd0 tx=0x7f94c000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 shutdown_connections 2024-09-19T09:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f94b40a32c0 0x7f94b40a5780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94c8101c30 0x7f94c8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94c81025f0 0x7f94c819f4a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 --2- 172.21.15.28:0/261298690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c8100a40 0x7f94c8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.675+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 >> 172.21.15.28:0/261298690 conn(0x7f94c80fc1c0 msgr2=0x7f94c8069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.676+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 shutdown_connections 2024-09-19T09:36:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:31.679+0000 7f94cf3c4640 1 -- 172.21.15.28:0/261298690 wait complete. 2024-09-19T09:36:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:32 smithi078 ceph-mon[35619]: pgmap v454: 97 pgs: 97 active+clean; 297 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-09-19T09:36:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:32 smithi038 ceph-mon[35271]: pgmap v454: 97 pgs: 97 active+clean; 297 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-09-19T09:36:33.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:32 smithi028 ceph-mon[30972]: pgmap v454: 97 pgs: 97 active+clean; 297 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-09-19T09:36:34.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:34 smithi078 ceph-mon[35619]: pgmap v455: 97 pgs: 97 active+clean; 292 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2024-09-19T09:36:34.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:34 smithi038 ceph-mon[35271]: pgmap v455: 97 pgs: 97 active+clean; 292 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2024-09-19T09:36:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:34 smithi028 ceph-mon[30972]: pgmap v455: 97 pgs: 97 active+clean; 292 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2024-09-19T09:36:36.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:36 smithi078 ceph-mon[35619]: pgmap v456: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 495 op/s 2024-09-19T09:36:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:36 smithi038 ceph-mon[35271]: pgmap v456: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 495 op/s 2024-09-19T09:36:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:36 smithi028 ceph-mon[30972]: pgmap v456: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 495 op/s 2024-09-19T09:36:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:38 smithi038 ceph-mon[35271]: pgmap v457: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 533 op/s 2024-09-19T09:36:39.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:38 smithi028 ceph-mon[30972]: pgmap v457: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 533 op/s 2024-09-19T09:36:39.039 INFO:tasks.workunit.client.0.smithi028.stdout: 98 1722464 529 1187285 581 847858 1475 2024-09-19T09:36:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:38 smithi078 ceph-mon[35619]: pgmap v457: 97 pgs: 97 active+clean; 282 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.3 MiB/s wr, 533 op/s 2024-09-19T09:36:40.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:40 smithi038 ceph-mon[35271]: pgmap v458: 97 pgs: 97 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 531 op/s 2024-09-19T09:36:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:40 smithi028 ceph-mon[30972]: pgmap v458: 97 pgs: 97 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 531 op/s 2024-09-19T09:36:41.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:40 smithi078 ceph-mon[35619]: pgmap v458: 97 pgs: 97 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 531 op/s 2024-09-19T09:36:41.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:41 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:36:42.021 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:41 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:36:42.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:41 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:36:42.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:42 smithi038 ceph-mon[35271]: pgmap v459: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 463 op/s 2024-09-19T09:36:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:42 smithi028 ceph-mon[30972]: pgmap v459: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 463 op/s 2024-09-19T09:36:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:42 smithi078 ceph-mon[35619]: pgmap v459: 97 pgs: 97 active+clean; 287 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 463 op/s 2024-09-19T09:36:44.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:44 smithi038 ceph-mon[35271]: pgmap v460: 97 pgs: 97 active+clean; 306 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-09-19T09:36:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 -- 172.21.15.28:0/1664260950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f8100a30 msgr2=0x7f31f8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/1664260950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f8100a30 0x7f31f8100e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009860 tx=0x7f31e802a340 comp rx=0 tx=0).stop 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 -- 172.21.15.28:0/1664260950 shutdown_connections 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/1664260950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f81025e0 0x7f31f8108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/1664260950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8101c20 0x7f31f81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/1664260950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f8100a30 0x7f31f8100e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 -- 172.21.15.28:0/1664260950 >> 172.21.15.28:0/1664260950 conn(0x7f31f80fc190 msgr2=0x7f31f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 -- 172.21.15.28:0/1664260950 shutdown_connections 2024-09-19T09:36:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.934+0000 7f31fc9e4640 1 -- 172.21.15.28:0/1664260950 wait complete. 2024-09-19T09:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 Processor -- start 2024-09-19T09:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 -- start start 2024-09-19T09:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8100a30 0x7f31f81049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 0x7f31f8103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31f810c690 con 0x7f31f8101c20 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31f810c510 con 0x7f31f81025e0 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31fc9e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31f810c810 con 0x7f31f8100a30 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 0x7f31f8103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.935+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8100a30 0x7f31f81049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 0x7f31f8103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41558/0 (socket says 172.21.15.28:41558) 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31effff640 1 -- 172.21.15.28:0/160086573 learned_addr learned my addr 172.21.15.28:0/160086573 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:50778/0 (socket says 172.21.15.28:50778) 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 -- 172.21.15.28:0/160086573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8100a30 msgr2=0x7f31f81049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8100a30 0x7f31f81049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 -- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 msgr2=0x7f31f8103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 0x7f31f8103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 -- 172.21.15.28:0/160086573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f8103dd0 con 0x7f31f81025e0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31f6d9d640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31f000efc0 tx=0x7f31f000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31ef7fe640 1 -- 172.21.15.28:0/160086573 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003970 con 0x7f31f81025e0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31ef7fe640 1 -- 172.21.15.28:0/160086573 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0004020 con 0x7f31f81025e0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31ef7fe640 1 -- 172.21.15.28:0/160086573 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f000b9f0 con 0x7f31f81025e0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.936+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f81a1ff0 con 0x7f31f81025e0 2024-09-19T09:36:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.937+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f81a2400 con 0x7f31f81025e0 2024-09-19T09:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.939+0000 7f31ef7fe640 1 -- 172.21.15.28:0/160086573 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f31f00041d0 con 0x7f31f81025e0 2024-09-19T09:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 Processor -- start 2024-09-19T09:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 -- start start 2024-09-19T09:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8d9a0 0x560d98d8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d93370 0x560d98d95830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560d98cf12f0 con 0x560d98d8d9a0 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d98c99270 con 0x560d98d93370 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31ff855740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d98ced2b0 con 0x560d98d8e2e0 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8d9a0 0x560d98d8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:50524/0 (socket says 172.21.15.28:50524) 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31f659c640 1 -- 172.21.15.28:0/2425094356 learned_addr learned my addr 172.21.15.28:0/2425094356 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.940+0000 7f31effff640 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d93370 0x560d98d95830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31ef7fe640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 0x7f31e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 -- 172.21.15.28:0/2425094356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d93370 msgr2=0x560d98d95830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d93370 0x560d98d95830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 -- 172.21.15.28:0/2425094356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8d9a0 msgr2=0x560d98d8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8d9a0 0x560d98d8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 -- 172.21.15.28:0/2425094356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98c98340 con 0x560d98d8e2e0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f6d9d640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 0x7f31e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31f659c640 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009830 tx=0x7f31e80095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=696a4ae3dd0e0648 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31ef7fe640 1 -- 172.21.15.28:0/160086573 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f31f00cfad0 con 0x7f31f81025e0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31ecff9640 1 -- 172.21.15.28:0/2425094356 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8037040 con 0x560d98d8e2e0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31ecff9640 1 -- 172.21.15.28:0/2425094356 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8042070 con 0x560d98d8e2e0 2024-09-19T09:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.941+0000 7f31ecff9640 1 -- 172.21.15.28:0/2425094356 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8005330 con 0x560d98d8e2e0 2024-09-19T09:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 -- 172.21.15.28:0/2425094356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 msgr2=0x560d98d92da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31f6d9d640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 0x7f31e40a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f31f0005fd0 tx=0x7f31f0066fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009830 tx=0x7f31e80095a0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 -- 172.21.15.28:0/2425094356 shutdown_connections 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d93370 0x560d98d95830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8e2e0 0x560d98d92da0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 --2- 172.21.15.28:0/2425094356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8d9a0 0x560d98d8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 -- 172.21.15.28:0/2425094356 >> 172.21.15.28:0/2425094356 conn(0x560d98d89010 msgr2=0x560d98d89410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.942+0000 7f31ff855740 1 -- 172.21.15.28:0/2425094356 shutdown_connections 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.943+0000 7f31ff855740 1 -- 172.21.15.28:0/2425094356 wait complete. 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.943+0000 7f31ff855740 1 -- start start 2024-09-19T09:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.943+0000 7f31ff855740 1 Processor -- start 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.943+0000 7f31ff855740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560d98d962e0 con 0x560d98d93370 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d98c7a290 con 0x560d98d8e2e0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d98d96460 con 0x560d98d8d9a0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ff855740 10 client.0 _ll_register_callbacks cb 0x560d98cab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:50804/0 (socket says 172.21.15.28:50804) 2024-09-19T09:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41576/0 (socket says 172.21.15.28:41576) 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31f659c640 1 -- 172.21.15.28:0/2520918253 learned_addr learned my addr 172.21.15.28:0/2520918253 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 -- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 msgr2=0x560d98d97b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d97b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 -- 172.21.15.28:0/2520918253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 msgr2=0x560d98d980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 -- 172.21.15.28:0/2520918253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98a04210 con 0x560d98d93370 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31effff640 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009620 tx=0x7f31e0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7e5dd1fa8c02d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.944+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0028060 con 0x560d98d93370 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f31e0011d60 con 0x560d98d93370 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e8042070 con 0x560d98d93370 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31e0013250 con 0x560d98d93370 2024-09-19T09:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 10 client.16485 fetch_fsmap learned FSMap version 34 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560d98ad0620 con 0x560d98d93370 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8042070 con 0x560d98d93370 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f31e0004b70 con 0x560d98d93370 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 10 client.16485 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 20 client.16485 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98da38a0 0x560d98da5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 4 client.16485 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d98cfea20 con 0x560d98da38a0 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.945+0000 7f31f659c640 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98da38a0 0x560d98da5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.946+0000 7f31ddffb640 10 client.16485 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.946+0000 7f31f659c640 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98da38a0 0x560d98da5d40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4eb29e5e276f17 server_cookie=77b2ac3b87c9feb9 in_seq=0 out_seq=0 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.948+0000 7f31ddffb640 1 -- 172.21.15.28:0/2520918253 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d98cfea20 con 0x560d98da38a0 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.948+0000 7f31ddffb640 10 client.16485 handle_command_reply: tid=0 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.948+0000 7f31ff855740 1 client.16485 shutdown 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 msgr2=0x560d98d9f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009620 tx=0x7f31e0009570 comp rx=0 tx=0).stop 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 shutdown_connections 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98da38a0 0x560d98da5d40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d93370 0x560d98d9f0e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ddffb640 0 client.16485 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 --2- 172.21.15.28:0/2520918253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d97b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 >> 172.21.15.28:0/2520918253 conn(0x560d98d89010 msgr2=0x560d98d8a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 shutdown_connections 2024-09-19T09:36:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 1 -- 172.21.15.28:0/2520918253 wait complete. 2024-09-19T09:36:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.949+0000 7f31ff855740 20 client.16485 trim_cache size 0 max 16384 2024-09-19T09:36:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.964+0000 7f31ff855740 1 Processor -- start 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 -- start start 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 0x560d98d96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8e2e0 0x560d98d89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81ab200 con 0x560d98d8e2e0 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81acdd0 con 0x560d98d89010 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31ff855740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81ab8d0 con 0x560d98d8d9a0 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 0x560d98d96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:50540/0 (socket says 172.21.15.28:50540) 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 -- 172.21.15.28:0/2591864110 learned_addr learned my addr 172.21.15.28:0/2591864110 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31effff640 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8e2e0 0x560d98d89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 -- 172.21.15.28:0/2591864110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 msgr2=0x560d98d96f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 0x560d98d96f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 -- 172.21.15.28:0/2591864110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8e2e0 msgr2=0x560d98d89e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8e2e0 0x560d98d89e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31f659c640 1 -- 172.21.15.28:0/2591864110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98da6320 con 0x560d98d8d9a0 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.965+0000 7f31effff640 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 0x560d98d96f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.966+0000 7f31f659c640 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31e802a930 tx=0x7f31e802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54003c966749ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.966+0000 7f31dffff640 1 -- 172.21.15.28:0/2591864110 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8037040 con 0x560d98d8d9a0 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.966+0000 7f31dffff640 1 -- 172.21.15.28:0/2591864110 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8042040 con 0x560d98d8d9a0 2024-09-19T09:36:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.966+0000 7f31dffff640 1 -- 172.21.15.28:0/2591864110 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80055c0 con 0x560d98d8d9a0 2024-09-19T09:36:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- 172.21.15.28:0/2591864110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 msgr2=0x560d98d9f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31e802a930 tx=0x7f31e802a340 comp rx=0 tx=0).stop 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- 172.21.15.28:0/2591864110 shutdown_connections 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d8e2e0 0x560d98d89e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9f000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- 172.21.15.28:0/2591864110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d89010 0x560d98d96f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- 172.21.15.28:0/2591864110 >> 172.21.15.28:0/2591864110 conn(0x560d98d92e90 msgr2=0x560d98d96800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- 172.21.15.28:0/2591864110 shutdown_connections 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- 172.21.15.28:0/2591864110 wait complete. 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- start start 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 Processor -- start 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d9dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81ac670 con 0x560d98d89010 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81abfa0 con 0x560d98d8e2e0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81a91e0 con 0x560d98d8d9a0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31ff855740 10 client.0 _ll_register_callbacks cb 0x7f31e0013ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.967+0000 7f31f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41588/0 (socket says 172.21.15.28:41588) 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 -- 172.21.15.28:0/2107361806 learned_addr learned my addr 172.21.15.28:0/2107361806 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d9dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 -- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 msgr2=0x560d98d9cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 -- 172.21.15.28:0/2107361806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 msgr2=0x560d98d9dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d9dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 -- 172.21.15.28:0/2107361806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98d9e260 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31f659c640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31effff640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009c70 tx=0x7f31e0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122d1cc456da470 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0030e60 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f31e0004030 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e8042040 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31e0004230 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ff855740 10 client.16491 fetch_fsmap learned FSMap version 34 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.968+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560d98e0e540 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e8042040 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f31e00048c0 con 0x560d98d89010 2024-09-19T09:36:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 10 client.16491 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 20 client.16491 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98d93370 0x560d98e10dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 4 client.16491 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d98cf8d70 con 0x560d98d93370 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31f659c640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98d93370 0x560d98e10dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31ddffb640 10 client.16491 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:44.969+0000 7f31f659c640 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98d93370 0x560d98e10dc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad23211a061a9e4d server_cookie=7cf16ccea879293e in_seq=0 out_seq=0 2024-09-19T09:36:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:44 smithi028 ceph-mon[30972]: pgmap v460: 97 pgs: 97 active+clean; 306 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-09-19T09:36:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:44 smithi078 ceph-mon[35619]: pgmap v460: 97 pgs: 97 active+clean; 306 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-09-19T09:36:46.555 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:36:46.555 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T09:36:46.556 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:36:46.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:46 smithi038 ceph-mon[35271]: pgmap v461: 97 pgs: 97 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-09-19T09:36:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:46 smithi028 ceph-mon[30972]: pgmap v461: 97 pgs: 97 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-09-19T09:36:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:46 smithi078 ceph-mon[35619]: pgmap v461: 97 pgs: 97 active+clean; 324 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-09-19T09:36:48.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:48 smithi038 ceph-mon[35271]: pgmap v462: 97 pgs: 97 active+clean; 353 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 491 op/s 2024-09-19T09:36:49.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:48 smithi028 ceph-mon[30972]: pgmap v462: 97 pgs: 97 active+clean; 353 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 491 op/s 2024-09-19T09:36:49.058 INFO:tasks.workunit.client.0.smithi028.stdout: 102 1286623 376 903083 509 633250 1184 2024-09-19T09:36:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:48 smithi078 ceph-mon[35619]: pgmap v462: 97 pgs: 97 active+clean; 353 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 491 op/s 2024-09-19T09:36:50.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:50 smithi038 ceph-mon[35271]: pgmap v463: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-09-19T09:36:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:50 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:36:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:50 smithi028 ceph-mon[30972]: pgmap v463: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-09-19T09:36:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:50 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:36:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:50 smithi078 ceph-mon[35619]: pgmap v463: 97 pgs: 97 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-09-19T09:36:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:50 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:36:51.172 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:36:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ddffb640 1 -- 172.21.15.28:0/2107361806 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d98cf8d70 con 0x560d98d93370 2024-09-19T09:36:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ddffb640 10 client.16491 handle_command_reply: tid=0 2024-09-19T09:36:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 client.16491 shutdown 2024-09-19T09:36:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 msgr2=0x560d98d9c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009c70 tx=0x7f31e0015040 comp rx=0 tx=0).stop 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 shutdown_connections 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d98d8e2e0 0x560d98d9dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x560d98d93370 0x560d98e10dc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d98d8d9a0 0x560d98d9cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 --2- 172.21.15.28:0/2107361806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560d98d89010 0x560d98d9c700 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.354+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 >> 172.21.15.28:0/2107361806 conn(0x560d98d92e90 msgr2=0x560d98d879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.355+0000 7f31ddffb640 0 client.16491 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.355+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 shutdown_connections 2024-09-19T09:36:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.357+0000 7f31ff855740 1 -- 172.21.15.28:0/2107361806 wait complete. 2024-09-19T09:36:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.358+0000 7f31ff855740 20 client.16491 trim_cache size 0 max 16384 2024-09-19T09:36:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:36:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:36:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:36:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:36:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.377+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 msgr2=0x7f31e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.377+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 0x7f31e40a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f31f0005fd0 tx=0x7f31f0066fa0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.377+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 msgr2=0x7f31f8103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.377+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31f000efc0 tx=0x7f31f000c800 comp rx=0 tx=0).stop 2024-09-19T09:36:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.378+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 shutdown_connections 2024-09-19T09:36:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.378+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f31e40a3240 0x7f31e40a5700 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.379+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31f81025e0 0x7f31f8103540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.379+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f8101c20 0x7f31f8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.379+0000 7f31fc9e4640 1 --2- 172.21.15.28:0/160086573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31f8100a30 0x7f31f81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.379+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 >> 172.21.15.28:0/160086573 conn(0x7f31f80fc190 msgr2=0x7f31f806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.382+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 shutdown_connections 2024-09-19T09:36:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:36:51.386+0000 7f31fc9e4640 1 -- 172.21.15.28:0/160086573 wait complete. 2024-09-19T09:36:52.004 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[70411]: *** Caught signal (Segmentation fault) ** 2024-09-19T09:36:52.004 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[70411]: in thread 7fd88389d640 thread_name:MR_Finisher 2024-09-19T09:36:52.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:52 smithi028 podman[96566]: 2024-09-19 09:36:52.003170125 +0000 UTC m=+0.091572635 container died 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T09:36:52.668 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:52 smithi028 ceph-mon[30972]: pgmap v464: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 981 op/s 2024-09-19T09:36:52.839 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:36:52.839 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"btime":"2024-09-19T09:36:40:611825+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":25804,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:36:39.981893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6837/558171377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":558171377},{"type":"v1","addr":"172.21.15.28:6837","nonce":558171377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 -- 172.21.15.28:0/749725418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c109e00 msgr2=0x7f9a4c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 --2- 172.21.15.28:0/749725418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c109e00 0x7f9a4c0742d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400afd0 tx=0x7f9a4402e720 comp rx=0 tx=0).stop 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 -- 172.21.15.28:0/749725418 shutdown_connections 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 --2- 172.21.15.28:0/749725418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a4c074990 0x7f9a4c07b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 --2- 172.21.15.28:0/749725418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c109e00 0x7f9a4c0742d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 --2- 172.21.15.28:0/749725418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 0x7f9a4c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 -- 172.21.15.28:0/749725418 >> 172.21.15.28:0/749725418 conn(0x7f9a4c06e140 msgr2=0x7f9a4c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 -- 172.21.15.28:0/749725418 shutdown_connections 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.185+0000 7f9a521e7640 1 -- 172.21.15.28:0/749725418 wait complete. 2024-09-19T09:36:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.186+0000 7f9a521e7640 1 Processor -- start 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 -- start start 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 0x7f9a4c076020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a4c076560 0x7f9a4c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c07e7e0 con 0x7f9a4c109450 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c07e660 con 0x7f9a4c076560 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.187+0000 7f9a521e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c07e960 con 0x7f9a4c074990 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.189+0000 7f9a4b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.189+0000 7f9a4b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:50576/0 (socket says 172.21.15.28:50576) 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.189+0000 7f9a4b7fe640 1 -- 172.21.15.28:0/435567185 learned_addr learned my addr 172.21.15.28:0/435567185 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.189+0000 7f9a4affd640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 0x7f9a4c076020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.190+0000 7f9a4b7fe640 1 -- 172.21.15.28:0/435567185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a4c076560 msgr2=0x7f9a4c076a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.190+0000 7f9a4b7fe640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a4c076560 0x7f9a4c076a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.190+0000 7f9a4b7fe640 1 -- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 msgr2=0x7f9a4c076020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.190+0000 7f9a4b7fe640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 0x7f9a4c076020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.190+0000 7f9a4b7fe640 1 -- 172.21.15.28:0/435567185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c137290 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.191+0000 7f9a4b7fe640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007bd0 tx=0x7f9a3c00f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.192+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00fc10 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.192+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c137520 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.192+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c137a00 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.192+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c00fdb0 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.192+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c003b50 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.196+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a38005290 con 0x7f9a4c074990 2024-09-19T09:36:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.198+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9a3c01c020 con 0x7f9a4c074990 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.200+0000 7f9a48ff9640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 0x7f9a340a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.200+0000 7f9a4affd640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 0x7f9a340a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.201+0000 7f9a4affd640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 0x7f9a340a5910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c077800 tx=0x7f9a4400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.202+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f9a3c0cecd0 con 0x7f9a4c074990 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.208+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a3c0d4020 con 0x7f9a4c074990 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.687+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a38005580 con 0x7f9a4c074990 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.690+0000 7f9a48ff9640 1 -- 172.21.15.28:0/435567185 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11955 (secure 0 0 0) 0x7f9a3c09b9c0 con 0x7f9a4c074990 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 msgr2=0x7f9a340a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 0x7f9a340a5910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c077800 tx=0x7f9a4400c040 comp rx=0 tx=0).stop 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 msgr2=0x7f9a4c07afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007bd0 tx=0x7f9a3c00f500 comp rx=0 tx=0).stop 2024-09-19T09:36:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 shutdown_connections 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9a340a3450 0x7f9a340a5910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a4c076560 0x7f9a4c076a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a4c109450 0x7f9a4c076020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 --2- 172.21.15.28:0/435567185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c074990 0x7f9a4c07afb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.697+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 >> 172.21.15.28:0/435567185 conn(0x7f9a4c06e140 msgr2=0x7f9a4c072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.698+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 shutdown_connections 2024-09-19T09:36:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:52.698+0000 7f9a521e7640 1 -- 172.21.15.28:0/435567185 wait complete. 2024-09-19T09:36:52.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:52 smithi038 ceph-mon[35271]: pgmap v464: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 981 op/s 2024-09-19T09:36:53.017 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:52 smithi028 podman[96566]: 2024-09-19 09:36:52.720541416 +0000 UTC m=+0.808943928 container remove 526b7f0d8e5c5094587e7fd195d047c500df7a8115a6dcebd5beaa537d8b1e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T09:36:53.017 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:52 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-09-19T09:36:53.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:52 smithi078 ceph-mon[35619]: pgmap v464: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 981 op/s 2024-09-19T09:36:54.016 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:53 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/435567185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:54.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:53 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/435567185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:54.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:53 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/435567185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:36:54.401 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:54 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-09-19T09:36:54.401 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:36:54 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Consumed 17.234s CPU time. 2024-09-19T09:36:54.404 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T09:36:54.405 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T09:36:54.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:54 smithi028 ceph-mon[30972]: pgmap v465: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 1.20k op/s 2024-09-19T09:36:55.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:54 smithi078 ceph-mon[35619]: pgmap v465: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 1.20k op/s 2024-09-19T09:36:55.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:54 smithi038 ceph-mon[35271]: pgmap v465: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 1.20k op/s 2024-09-19T09:36:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:56 smithi028 ceph-mon[30972]: pgmap v466: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 1.13k op/s 2024-09-19T09:36:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:56 smithi078 ceph-mon[35619]: pgmap v466: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 1.13k op/s 2024-09-19T09:36:57.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:56 smithi038 ceph-mon[35271]: pgmap v466: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 1.13k op/s 2024-09-19T09:36:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:36:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:58 smithi028 ceph-mon[30972]: pgmap v467: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2024-09-19T09:36:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:36:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:36:59.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:36:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:36:59.057 INFO:tasks.workunit.client.0.smithi028.stdout: 113 1962573 461 1431236 618 998646 1900 2024-09-19T09:36:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:58 smithi078 ceph-mon[35619]: pgmap v467: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2024-09-19T09:36:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:36:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:36:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:36:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:36:59.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:58 smithi038 ceph-mon[35271]: pgmap v467: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2024-09-19T09:36:59.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:36:59.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:36:59.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:36:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:36:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 -- 172.21.15.28:0/1553790716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 msgr2=0x7f756c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 --2- 172.21.15.28:0/1553790716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c10bbb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c02a480 comp rx=0 tx=0).stop 2024-09-19T09:36:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 -- 172.21.15.28:0/1553790716 shutdown_connections 2024-09-19T09:36:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 --2- 172.21.15.28:0/1553790716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 --2- 172.21.15.28:0/1553790716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c10bbb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 --2- 172.21.15.28:0/1553790716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 -- 172.21.15.28:0/1553790716 >> 172.21.15.28:0/1553790716 conn(0x7f756c0fc7c0 msgr2=0x7f756c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 -- 172.21.15.28:0/1553790716 shutdown_connections 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.862+0000 7f75735b7640 1 -- 172.21.15.28:0/1553790716 wait complete. 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 Processor -- start 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 -- start start 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f756c111f60 con 0x7f756c1036c0 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f756c111de0 con 0x7f756c10c180 2024-09-19T09:36:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f75735b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1120e0 con 0x7f756c102da0 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f7570b52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f7570b52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54586/0 (socket says 172.21.15.28:54586) 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f7571353640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f7570b52640 1 -- 172.21.15.28:0/2834711506 learned_addr learned my addr 172.21.15.28:0/2834711506 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.863+0000 7f7571b54640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 -- 172.21.15.28:0/2834711506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 msgr2=0x7f756c100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 -- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 msgr2=0x7f756c100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 -- 172.21.15.28:0/2834711506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c1a6920 con 0x7f756c10c180 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571353640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f7571b54640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c101480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7564009fd0 tx=0x7f756400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f75627fc640 1 -- 172.21.15.28:0/2834711506 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400ce20 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f75627fc640 1 -- 172.21.15.28:0/2834711506 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7564005020 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f75627fc640 1 -- 172.21.15.28:0/2834711506 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564010720 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c1a6c10 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.864+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a70a0 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.866+0000 7f75627fc640 1 -- 172.21.15.28:0/2834711506 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7564005540 con 0x7f756c10c180 2024-09-19T09:36:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.867+0000 7f75627fc640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 0x7f75580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.867+0000 7f7575990740 1 Processor -- start 2024-09-19T09:36:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f75627fc640 1 -- 172.21.15.28:0/2834711506 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f75640cff00 con 0x7f756c10c180 2024-09-19T09:36:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 0x7f75580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 -- start start 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x559992401400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992403d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55999235c2a0 con 0x5599923f7080 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559992303b10 con 0x559992401940 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7575990740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559992358260 con 0x5599923f6760 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 0x7f75580a5780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fc0 tx=0x7f7568005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:59780/0 (socket says 172.21.15.28:59780) 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 -- 172.21.15.28:0/764539052 learned_addr learned my addr 172.21.15.28:0/764539052 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571b54640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992403d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7570b52640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x559992401400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 -- 172.21.15.28:0/764539052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 msgr2=0x559992403d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992403d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.868+0000 7f7571353640 1 -- 172.21.15.28:0/764539052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 msgr2=0x559992401400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f7571353640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x559992401400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f7571353640 1 -- 172.21.15.28:0/764539052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599920cdb90 con 0x5599923f6760 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f7570b52640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x559992401400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:36:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f7571353640 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75680042c0 tx=0x7f7568014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83b34b1ad176daf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f75577fe640 1 -- 172.21.15.28:0/764539052 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800f480 con 0x5599923f6760 2024-09-19T09:36:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f75577fe640 1 -- 172.21.15.28:0/764539052 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756800fb00 con 0x5599923f6760 2024-09-19T09:36:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.869+0000 7f75577fe640 1 -- 172.21.15.28:0/764539052 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756801dc20 con 0x5599923f6760 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- 172.21.15.28:0/764539052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 msgr2=0x5599923f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75680042c0 tx=0x7f7568014000 comp rx=0 tx=0).stop 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- 172.21.15.28:0/764539052 shutdown_connections 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992403d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x559992401400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 --2- 172.21.15.28:0/764539052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923f6b40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- 172.21.15.28:0/764539052 >> 172.21.15.28:0/764539052 conn(0x5599923f2cf0 msgr2=0x5599923f30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- 172.21.15.28:0/764539052 shutdown_connections 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- 172.21.15.28:0/764539052 wait complete. 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.870+0000 7f7575990740 1 -- start start 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7575990740 1 Processor -- start 2024-09-19T09:36:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7575990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7575990740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x5599923fec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7575990740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7570b52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x5599923fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7571353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7571353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:37790/0 (socket says 172.21.15.28:37790) 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7570b52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x5599923fec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:59782/0 (socket says 172.21.15.28:59782) 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7571b54640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.871+0000 7f7571b54640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54600/0 (socket says 172.21.15.28:54600) 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55999240cdc0 con 0x559992401940 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 -- 172.21.15.28:0/4164225605 learned_addr learned my addr 172.21.15.28:0/4164225605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5599922e46e0 con 0x5599923f6760 2024-09-19T09:36:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55999240cf40 con 0x5599923f7080 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 -- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 msgr2=0x5599923fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x5599923fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 -- 172.21.15.28:0/4164225605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 msgr2=0x5599923ff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559992416230 con 0x5599923f6760 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571b54640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7575990740 10 client.0 _ll_register_callbacks cb 0x559992308a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756800fb00 con 0x5599923f6760 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7571353640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7568003e50 tx=0x7f756801d500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc74886d5badbe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.872+0000 7f7554ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7554ff9640 1 -- 172.21.15.28:0/4164225605 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800fcb0 con 0x5599923f6760 2024-09-19T09:36:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7554ff9640 1 -- 172.21.15.28:0/4164225605 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7568009040 con 0x5599923f6760 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7554ff9640 1 -- 172.21.15.28:0/4164225605 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f756800fe50 con 0x5599923f6760 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 10 client.26078 fetch_fsmap learned FSMap version 34 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=34}) -- 0x559992418910 con 0x5599923f6760 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7554ff9640 1 -- 172.21.15.28:0/4164225605 <== mon.2 v2:172.21.15.38:3300/0 4 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f756801a070 con 0x5599923f6760 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 10 client.26078 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 10 client.26078 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 20 client.26078 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992419cc0 0x55999241c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 4 client.26078 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.873+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f756800fb00 con 0x559992419cc0 2024-09-19T09:36:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.874+0000 7f7570b52640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992419cc0 0x55999241c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.874+0000 7f7570b52640 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992419cc0 0x55999241c0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6561e95268ed479 server_cookie=d5ecdb0e8a93299d in_seq=0 out_seq=0 2024-09-19T09:36:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.874+0000 7f7554ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.877+0000 7f7554ff9640 1 -- 172.21.15.28:0/4164225605 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f756800fb00 con 0x559992419cc0 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.877+0000 7f7554ff9640 10 client.26078 handle_command_reply: tid=0 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.877+0000 7f7575990740 1 client.26078 shutdown 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 msgr2=0x5599923fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7568003e50 tx=0x7f756801d500 comp rx=0 tx=0).stop 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 shutdown_connections 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992419cc0 0x55999241c0b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559992401940 0x5599923ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x5599923fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7554ff9640 0 client.26078 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 --2- 172.21.15.28:0/4164225605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599923f6760 0x5599923fe750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 >> 172.21.15.28:0/4164225605 conn(0x5599923f2cf0 msgr2=0x5599923f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 shutdown_connections 2024-09-19T09:36:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 1 -- 172.21.15.28:0/4164225605 wait complete. 2024-09-19T09:36:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.878+0000 7f7575990740 20 client.26078 trim_cache size 0 max 16384 2024-09-19T09:36:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 Processor -- start 2024-09-19T09:36:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 -- start start 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992421d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ab2d0 con 0x5599923f7080 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1acea0 con 0x559992401940 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7575990740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ab9a0 con 0x5599923f6760 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54612/0 (socket says 172.21.15.28:54612) 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 -- 172.21.15.28:0/1389794781 learned_addr learned my addr 172.21.15.28:0/1389794781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7571353640 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7571b54640 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992421d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 -- 172.21.15.28:0/1389794781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 msgr2=0x5599923fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 -- 172.21.15.28:0/1389794781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 msgr2=0x559992421d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992421d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 -- 172.21.15.28:0/1389794781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599923f2b20 con 0x5599923f7080 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.897+0000 7f7570b52640 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f755c031550 tx=0x7f755c031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4185064802d7079 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7556ffd640 1 -- 172.21.15.28:0/1389794781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c04a070 con 0x5599923f7080 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7556ffd640 1 -- 172.21.15.28:0/1389794781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f755c004400 con 0x5599923f7080 2024-09-19T09:36:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7556ffd640 1 -- 172.21.15.28:0/1389794781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c035040 con 0x5599923f7080 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 -- 172.21.15.28:0/1389794781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 msgr2=0x5599924217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f755c031550 tx=0x7f755c031490 comp rx=0 tx=0).stop 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 -- 172.21.15.28:0/1389794781 shutdown_connections 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x559992421d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f7080 0x5599924217d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 --2- 172.21.15.28:0/1389794781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f6760 0x5599923fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 -- 172.21.15.28:0/1389794781 >> 172.21.15.28:0/1389794781 conn(0x5599923f2cf0 msgr2=0x5599923fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 -- 172.21.15.28:0/1389794781 shutdown_connections 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.898+0000 7f7575990740 1 -- 172.21.15.28:0/1389794781 wait complete. 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 -- start start 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 Processor -- start 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x55999240ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x55999247d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ac740 con 0x5599923f6760 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ac070 con 0x559992401940 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1a92b0 con 0x5599923f7080 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7571353640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7575990740 10 client.0 _ll_register_callbacks cb 0x7f756801d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7571353640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54618/0 (socket says 172.21.15.28:54618) 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.899+0000 7f7571353640 1 -- 172.21.15.28:0/1608140365 learned_addr learned my addr 172.21.15.28:0/1608140365 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7570b52640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x55999240ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571b54640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x55999247d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 -- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 msgr2=0x55999240ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x55999240ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 -- 172.21.15.28:0/1608140365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 msgr2=0x55999247d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x55999247d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:36:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 -- 172.21.15.28:0/1608140365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599923f3da0 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7570b52640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x55999240ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7571353640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7568005670 tx=0x7f7568018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247785fdb4346bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680256e0 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f7568025880 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755c004400 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7568025a80 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7575990740 10 client.16512 fetch_fsmap learned FSMap version 34 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.900+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559992484ff0 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f755c004400 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f756801a040 con 0x5599923f6760 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 10 client.16512 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:36:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 10 client.16512 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 20 client.16512 populate_metadata read hostname 'smithi028' 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992485660 0x559992487a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 4 client.16512 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x5599923699d0 con 0x559992485660 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7570b52640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992485660 0x559992487a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7554ff9640 10 client.16512 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:36:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:36:59.901+0000 7f7570b52640 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992485660 0x559992487a50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f59c52cffdb7501 server_cookie=58beb5933e0da11 in_seq=0 out_seq=0 2024-09-19T09:37:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:00 smithi028 ceph-mon[30972]: pgmap v468: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 923 op/s 2024-09-19T09:37:01.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:00 smithi078 ceph-mon[35619]: pgmap v468: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 923 op/s 2024-09-19T09:37:01.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:00 smithi038 ceph-mon[35271]: pgmap v468: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 923 op/s 2024-09-19T09:37:02.977 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:02 smithi038 ceph-mon[35271]: pgmap v469: 97 pgs: 97 active+clean; 335 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.1 MiB/s wr, 600 op/s 2024-09-19T09:37:03.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:02 smithi028 ceph-mon[30972]: pgmap v469: 97 pgs: 97 active+clean; 335 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.1 MiB/s wr, 600 op/s 2024-09-19T09:37:03.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:02 smithi078 ceph-mon[35619]: pgmap v469: 97 pgs: 97 active+clean; 335 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.1 MiB/s wr, 600 op/s 2024-09-19T09:37:04.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-09-19T09:37:04.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 systemd[1]: Stopped Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:37:04.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Consumed 17.234s CPU time. 2024-09-19T09:37:04.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 systemd[1]: Starting Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:37:04.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 podman[96879]: 2024-09-19 09:37:04.399092135 +0000 UTC m=+0.018612564 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 -- 172.21.15.28:0/4273188829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c100a40 msgr2=0x7f2a1c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 --2- 172.21.15.28:0/4273188829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c100a40 0x7f2a1c100e40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009ca0 tx=0x7f2a0c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 -- 172.21.15.28:0/4273188829 shutdown_connections 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 --2- 172.21.15.28:0/4273188829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 --2- 172.21.15.28:0/4273188829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c101c30 0x7f2a1c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 --2- 172.21.15.28:0/4273188829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c100a40 0x7f2a1c100e40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 -- 172.21.15.28:0/4273188829 >> 172.21.15.28:0/4273188829 conn(0x7f2a1c0fc1c0 msgr2=0x7f2a1c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 -- 172.21.15.28:0/4273188829 shutdown_connections 2024-09-19T09:37:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.571+0000 7f2a23991640 1 -- 172.21.15.28:0/4273188829 wait complete. 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 Processor -- start 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 -- start start 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 0x7f2a1c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c101c30 0x7f2a1c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c10c6a0 con 0x7f2a1c1025f0 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c10c520 con 0x7f2a1c101c30 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a23991640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c10c820 con 0x7f2a1c100a40 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 0x7f2a1c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53594/0 (socket says 172.21.15.28:53594) 2024-09-19T09:37:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 -- 172.21.15.28:0/4062865482 learned_addr learned my addr 172.21.15.28:0/4062865482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 0x7f2a1c06a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:39808/0 (socket says 172.21.15.28:39808) 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c101c30 0x7f2a1c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 -- 172.21.15.28:0/4062865482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 msgr2=0x7f2a1c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 0x7f2a1c06a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 -- 172.21.15.28:0/4062865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c101c30 msgr2=0x7f2a1c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c101c30 0x7f2a1c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 -- 172.21.15.28:0/4062865482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c19fad0 con 0x7f2a1c1025f0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.572+0000 7f2a21f2e640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c970 tx=0x7f2a1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.573+0000 7f2a1a7fc640 1 -- 172.21.15.28:0/4062865482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007bb0 con 0x7f2a1c1025f0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.573+0000 7f2a1a7fc640 1 -- 172.21.15.28:0/4062865482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a14007d50 con 0x7f2a1c1025f0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.573+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c19fd60 con 0x7f2a1c1025f0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.573+0000 7f2a1a7fc640 1 -- 172.21.15.28:0/4062865482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14018800 con 0x7f2a1c1025f0 2024-09-19T09:37:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.573+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1a0200 con 0x7f2a1c1025f0 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.575+0000 7f2a1a7fc640 1 -- 172.21.15.28:0/4062865482 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2a1400fa50 con 0x7f2a1c1025f0 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.575+0000 7f2a2680b740 1 Processor -- start 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.575+0000 7f2a2680b740 1 -- start start 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c95bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 0x56358c963830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56358c8bf2f0 con 0x56358c95b9a0 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56358c867270 con 0x56358c95c2e0 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2680b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56358c8bb2b0 con 0x56358c961370 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a1a7fc640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 0x7f2a040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c95bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c95bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53608/0 (socket says 172.21.15.28:53608) 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:39656/0 (socket says 172.21.15.28:39656) 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a20f2c640 1 -- 172.21.15.28:0/3191403493 learned_addr learned my addr 172.21.15.28:0/3191403493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 0x56358c963830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a21f2e640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 0x7f2a040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a1a7fc640 1 -- 172.21.15.28:0/4062865482 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2a140cf810 con 0x7f2a1c1025f0 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 -- 172.21.15.28:0/3191403493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 msgr2=0x56358c963830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 0x56358c963830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 -- 172.21.15.28:0/3191403493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 msgr2=0x56358c95bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c95bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 -- 172.21.15.28:0/3191403493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56358c866340 con 0x56358c95c2e0 2024-09-19T09:37:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 0x56358c963830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009bf0 tx=0x7f2a0c005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98731deaf9984ed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.576+0000 7f2a21f2e640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 0x7f2a040a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a140982c0 tx=0x7f2a140bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a037fe640 1 -- 172.21.15.28:0/3191403493 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c049070 con 0x56358c95c2e0 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a037fe640 1 -- 172.21.15.28:0/3191403493 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c036040 con 0x56358c95c2e0 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a037fe640 1 -- 172.21.15.28:0/3191403493 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c02fe40 con 0x56358c95c2e0 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 -- 172.21.15.28:0/3191403493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 msgr2=0x56358c960da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009bf0 tx=0x7f2a0c005fa0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 -- 172.21.15.28:0/3191403493 shutdown_connections 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95c2e0 0x56358c960da0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c961370 0x56358c963830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3191403493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c95bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 -- 172.21.15.28:0/3191403493 >> 172.21.15.28:0/3191403493 conn(0x56358c957010 msgr2=0x56358c957410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 -- 172.21.15.28:0/3191403493 shutdown_connections 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.577+0000 7f2a2680b740 1 -- 172.21.15.28:0/3191403493 wait complete. 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 -- start start 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 Processor -- start 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95b9a0 0x56358c9659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c965f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 0x56358c96cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56358c964160 con 0x56358c961370 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56358c848290 con 0x56358c95b9a0 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56358c9642e0 con 0x56358c95c2e0 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2680b740 10 client.0 _ll_register_callbacks cb 0x56358c879140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95b9a0 0x56358c9659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c965f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 0x56358c96cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c965f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:39822/0 (socket says 172.21.15.28:39822) 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.578+0000 7f2a2172d640 1 -- 172.21.15.28:0/2300832939 learned_addr learned my addr 172.21.15.28:0/2300832939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 -- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 msgr2=0x56358c965f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c965f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 -- 172.21.15.28:0/2300832939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95b9a0 msgr2=0x56358c9659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95b9a0 0x56358c9659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 -- 172.21.15.28:0/2300832939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56358c5d2210 con 0x56358c961370 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 0x56358c96cf40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x56358c964ef0 tx=0x7f2a1000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bae6c52a20e255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001b040 con 0x56358c961370 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2a10011e40 con 0x56358c961370 2024-09-19T09:37:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c036040 con 0x56358c961370 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2a10005020 con 0x56358c961370 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a2680b740 10 client.16524 fetch_fsmap learned FSMap version 34 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56358c69e620 con 0x56358c961370 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.579+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c036040 con 0x56358c961370 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2a10004cc0 con 0x56358c961370 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 10 client.16524 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 20 client.16524 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c973bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 4 client.16524 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56358c8cca20 con 0x56358c971750 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2172d640 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c973bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a2172d640 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c973bb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c64d3d3ad8cd130 server_cookie=14d4a98258371e3e in_seq=0 out_seq=0 2024-09-19T09:37:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.580+0000 7f2a00ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.582+0000 7f2a00ff9640 1 -- 172.21.15.28:0/2300832939 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56358c8cca20 con 0x56358c971750 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.582+0000 7f2a00ff9640 10 client.16524 handle_command_reply: tid=0 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 client.16524 shutdown 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 msgr2=0x56358c96cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 0x56358c96cf40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x56358c964ef0 tx=0x7f2a1000b410 comp rx=0 tx=0).stop 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 shutdown_connections 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c973bb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c961370 0x56358c96cf40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a00ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c965f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 --2- 172.21.15.28:0/2300832939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c95b9a0 0x56358c9659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 >> 172.21.15.28:0/2300832939 conn(0x56358c957010 msgr2=0x56358c958820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 shutdown_connections 2024-09-19T09:37:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 1 -- 172.21.15.28:0/2300832939 wait complete. 2024-09-19T09:37:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.583+0000 7f2a2680b740 20 client.16524 trim_cache size 0 max 16384 2024-09-19T09:37:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.597+0000 7f2a2680b740 1 Processor -- start 2024-09-19T09:37:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 -- start start 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95b9a0 0x56358c964f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c957e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1a8e10 con 0x56358c95c2e0 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1aa9e0 con 0x56358c961370 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2680b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1a94e0 con 0x56358c95b9a0 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53624/0 (socket says 172.21.15.28:53624) 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 -- 172.21.15.28:0/3508960385 learned_addr learned my addr 172.21.15.28:0/3508960385 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95b9a0 0x56358c964f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c957e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 -- 172.21.15.28:0/3508960385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95b9a0 msgr2=0x56358c964f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95b9a0 0x56358c964f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 -- 172.21.15.28:0/3508960385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 msgr2=0x56358c957e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c957e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 -- 172.21.15.28:0/3508960385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56358c965500 con 0x56358c95c2e0 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c957e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.598+0000 7f2a2172d640 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c031ae0 tx=0x7f2a0c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f228fd18faffec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a02ffd640 1 -- 172.21.15.28:0/3508960385 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c049040 con 0x56358c95c2e0 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a02ffd640 1 -- 172.21.15.28:0/3508960385 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c036040 con 0x56358c95c2e0 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a02ffd640 1 -- 172.21.15.28:0/3508960385 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0318d0 con 0x56358c95c2e0 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 -- 172.21.15.28:0/3508960385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 msgr2=0x56358c96cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c031ae0 tx=0x7f2a0c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 -- 172.21.15.28:0/3508960385 shutdown_connections 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c957e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95c2e0 0x56358c96cf00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 --2- 172.21.15.28:0/3508960385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95b9a0 0x56358c964f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 -- 172.21.15.28:0/3508960385 >> 172.21.15.28:0/3508960385 conn(0x56358c957010 msgr2=0x56358c9659c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 -- 172.21.15.28:0/3508960385 shutdown_connections 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.599+0000 7f2a2680b740 1 -- 172.21.15.28:0/3508960385 wait complete. 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 -- start start 2024-09-19T09:37:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 Processor -- start 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c96a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c9da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1aa280 con 0x56358c95b9a0 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1a9bb0 con 0x56358c961370 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1a6df0 con 0x56358c95c2e0 2024-09-19T09:37:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2680b740 10 client.0 _ll_register_callbacks cb 0x7f2a10012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53634/0 (socket says 172.21.15.28:53634) 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 -- 172.21.15.28:0/4095015099 learned_addr learned my addr 172.21.15.28:0/4095015099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a2172d640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c96a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c9da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 -- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 msgr2=0x56358c96a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.600+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c96a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a20f2c640 1 -- 172.21.15.28:0/4095015099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 msgr2=0x56358c9da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c9da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a20f2c640 1 -- 172.21.15.28:0/4095015099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56358c9daa90 con 0x56358c95b9a0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a20f2c640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005fd0 tx=0x7f2a1000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2fe187e6429789 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000b9b0 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8182+0+0 (secure 0 0 0) 0x7f2a100055d0 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c036040 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2a10041070 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 10 client.16530 fetch_fsmap learned FSMap version 34 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56358c9dbde0 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c036040 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2a10038020 con 0x56358c95b9a0 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 10 client.16530 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 20 client.16530 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c9de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 4 client.16530 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:37:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.601+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56358c8c6d70 con 0x56358c971750 2024-09-19T09:37:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.602+0000 7f2a2172d640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c9de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.602+0000 7f2a00ff9640 10 client.16530 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:04.602+0000 7f2a2172d640 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c9de840 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1230fb7752d72b5b server_cookie=e68a1751cddd30ab in_seq=0 out_seq=0 2024-09-19T09:37:04.788 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:04 smithi028 ceph-mon[30972]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-09-19T09:37:04.788 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 podman[96879]: 2024-09-19 09:37:04.515498627 +0000 UTC m=+0.135019046 container create 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T09:37:04.788 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 podman[96879]: 2024-09-19 09:37:04.760306733 +0000 UTC m=+0.379827169 container init 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T09:37:04.788 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 podman[96879]: 2024-09-19 09:37:04.763314669 +0000 UTC m=+0.382835090 container start 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T09:37:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:04 smithi078 ceph-mon[35619]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-09-19T09:37:05.175 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:04 smithi038 ceph-mon[35271]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 3.3 MiB/s wr, 489 op/s 2024-09-19T09:37:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 ceph-mds[96943]: -- 172.21.15.28:0/4194588042 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4ee3adc20 con 0x55b4ee38b000 2024-09-19T09:37:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: starting mds.d at 2024-09-19T09:37:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 bash[96879]: 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 2024-09-19T09:37:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:37:04 smithi028 systemd[1]: Started Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a00ff9640 1 -- 172.21.15.28:0/4095015099 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56358c8c6d70 con 0x56358c971750 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a00ff9640 10 client.16530 handle_command_reply: tid=0 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 client.16530 shutdown 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 msgr2=0x56358c969f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005fd0 tx=0x7f2a1000b410 comp rx=0 tx=0).stop 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 shutdown_connections 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56358c961370 0x56358c9da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56358c971750 0x56358c9de840 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56358c95c2e0 0x56358c96a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 --2- 172.21.15.28:0/4095015099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56358c95b9a0 0x56358c969f40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 >> 172.21.15.28:0/4095015099 conn(0x56358c957010 msgr2=0x56358c9559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.676+0000 7f2a00ff9640 0 client.16530 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 shutdown_connections 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a2680b740 1 -- 172.21.15.28:0/4095015099 wait complete. 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a2680b740 20 client.16530 trim_cache size 0 max 16384 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:37:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:37:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:37:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:37:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 msgr2=0x7f2a040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 0x7f2a040a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a140982c0 tx=0x7f2a140bc040 comp rx=0 tx=0).stop 2024-09-19T09:37:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 msgr2=0x7f2a1c19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c970 tx=0x7f2a1400ce30 comp rx=0 tx=0).stop 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 shutdown_connections 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f2a040a3240 0x7f2a040a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a1c101c30 0x7f2a1c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a1c1025f0 0x7f2a1c19f1c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 --2- 172.21.15.28:0/4062865482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a1c100a40 0x7f2a1c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 >> 172.21.15.28:0/4062865482 conn(0x7f2a1c0fc1c0 msgr2=0x7f2a1c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 shutdown_connections 2024-09-19T09:37:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:05.677+0000 7f2a23991640 1 -- 172.21.15.28:0/4062865482 wait complete. 2024-09-19T09:37:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:05 smithi028 ceph-mon[30972]: daemon mds.d restarted 2024-09-19T09:37:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:05 smithi028 ceph-mon[30972]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:37:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:05 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6836/4201169804,v1:172.21.15.28:6837/4201169804] up:boot 2024-09-19T09:37:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:05 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:37:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:37:06.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:05 smithi078 ceph-mon[35619]: daemon mds.d restarted 2024-09-19T09:37:06.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:05 smithi078 ceph-mon[35619]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:37:06.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:05 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6836/4201169804,v1:172.21.15.28:6837/4201169804] up:boot 2024-09-19T09:37:06.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:05 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:37:06.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:37:06.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:05 smithi038 ceph-mon[35271]: daemon mds.d restarted 2024-09-19T09:37:06.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:05 smithi038 ceph-mon[35271]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:37:06.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:05 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6836/4201169804,v1:172.21.15.28:6837/4201169804] up:boot 2024-09-19T09:37:06.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:05 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:37:06.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:37:07.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:07 smithi028 ceph-mon[30972]: pgmap v472: 97 pgs: 97 active+clean; 373 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 464 op/s 2024-09-19T09:37:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:07 smithi078 ceph-mon[35619]: pgmap v472: 97 pgs: 97 active+clean; 373 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 464 op/s 2024-09-19T09:37:07.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:07 smithi038 ceph-mon[35271]: pgmap v472: 97 pgs: 97 active+clean; 373 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 464 op/s 2024-09-19T09:37:09.058 INFO:tasks.workunit.client.0.smithi028.stdout: 114 196198 27 140304 89 103630 153 2024-09-19T09:37:09.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:09 smithi028 ceph-mon[30972]: pgmap v473: 97 pgs: 97 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 1.03k op/s 2024-09-19T09:37:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:09 smithi078 ceph-mon[35619]: pgmap v473: 97 pgs: 97 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 1.03k op/s 2024-09-19T09:37:09.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:09 smithi038 ceph-mon[35271]: pgmap v473: 97 pgs: 97 active+clean; 386 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.9 MiB/s wr, 1.03k op/s 2024-09-19T09:37:11.517 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:11 smithi028 ceph-mon[30972]: pgmap v474: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 1.60k op/s 2024-09-19T09:37:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:11 smithi078 ceph-mon[35619]: pgmap v474: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 1.60k op/s 2024-09-19T09:37:11.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:11 smithi038 ceph-mon[35271]: pgmap v474: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 1.60k op/s 2024-09-19T09:37:13.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:13 smithi028 ceph-mon[30972]: pgmap v475: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.3 MiB/s wr, 1.68k op/s 2024-09-19T09:37:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:13 smithi078 ceph-mon[35619]: pgmap v475: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.3 MiB/s wr, 1.68k op/s 2024-09-19T09:37:13.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:13 smithi038 ceph-mon[35271]: pgmap v475: 97 pgs: 97 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.3 MiB/s wr, 1.68k op/s 2024-09-19T09:37:15.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:15 smithi028 ceph-mon[30972]: pgmap v476: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 1.67k op/s 2024-09-19T09:37:15.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:15 smithi078 ceph-mon[35619]: pgmap v476: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 1.67k op/s 2024-09-19T09:37:15.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:15 smithi038 ceph-mon[35271]: pgmap v476: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 1.67k op/s 2024-09-19T09:37:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7554ff9640 1 -- 172.21.15.28:0/1608140365 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5599923699d0 con 0x559992485660 2024-09-19T09:37:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7554ff9640 10 client.16512 handle_command_reply: tid=0 2024-09-19T09:37:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 client.16512 shutdown 2024-09-19T09:37:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 msgr2=0x55999240e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7568005670 tx=0x7f7568018000 comp rx=0 tx=0).stop 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 shutdown_connections 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559992401940 0x55999247d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559992485660 0x559992487a50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599923f7080 0x55999240ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 --2- 172.21.15.28:0/1608140365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5599923f6760 0x55999240e7c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7554ff9640 0 client.16512 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 >> 172.21.15.28:0/1608140365 conn(0x5599923f2cf0 msgr2=0x5599923f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 shutdown_connections 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 1 -- 172.21.15.28:0/1608140365 wait complete. 2024-09-19T09:37:15.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.911+0000 7f7575990740 20 client.16512 trim_cache size 0 max 16384 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 35, 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 18, 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "c9c0d8a": { 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 18, 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:37:15.916 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 149.8, 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:15.917 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.912+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 msgr2=0x7f75580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.912+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 0x7f75580a5780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fc0 tx=0x7f7568005f90 comp rx=0 tx=0).stop 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.912+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 msgr2=0x7f756c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.912+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c101480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7564009fd0 tx=0x7f756400ef10 comp rx=0 tx=0).stop 2024-09-19T09:37:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.912+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 shutdown_connections 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f75580a32c0 0x7f75580a5780 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f756c10c180 0x7f756c101480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756c1036c0 0x7f756c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 --2- 172.21.15.28:0/2834711506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f756c102da0 0x7f756c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 >> 172.21.15.28:0/2834711506 conn(0x7f756c0fc7c0 msgr2=0x7f756c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 shutdown_connections 2024-09-19T09:37:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:15.913+0000 7f75735b7640 1 -- 172.21.15.28:0/2834711506 wait complete. 2024-09-19T09:37:16.676 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c9c0d8a', quiesce took 30.1 seconds. Will release after: -0.3000000000000007 2024-09-19T09:37:16.676 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:37:16.949 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:17.423 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:17 smithi028 ceph-mon[30972]: pgmap v477: 97 pgs: 97 active+clean; 384 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 1.44k op/s 2024-09-19T09:37:17.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:17 smithi078 ceph-mon[35619]: pgmap v477: 97 pgs: 97 active+clean; 384 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 1.44k op/s 2024-09-19T09:37:17.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 -- 172.21.15.28:0/2165889412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca41027a0 msgr2=0x7f7ca41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 --2- 172.21.15.28:0/2165889412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca41027a0 0x7f7ca41067f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009900 tx=0x7f7c9002a410 comp rx=0 tx=0).stop 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 -- 172.21.15.28:0/2165889412 shutdown_connections 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 --2- 172.21.15.28:0/2165889412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4106f20 0x7f7ca410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 --2- 172.21.15.28:0/2165889412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca41027a0 0x7f7ca41067f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 --2- 172.21.15.28:0/2165889412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca4101df0 0x7f7ca41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.620+0000 7f7caae57640 1 -- 172.21.15.28:0/2165889412 >> 172.21.15.28:0/2165889412 conn(0x7f7ca40fd8f0 msgr2=0x7f7ca40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- 172.21.15.28:0/2165889412 shutdown_connections 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- 172.21.15.28:0/2165889412 wait complete. 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 Processor -- start 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- start start 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca41027a0 0x7f7ca4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:17.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 0x7f7ca419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4111f40 con 0x7f7ca4106f20 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4111dc0 con 0x7f7ca41027a0 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.621+0000 7f7caae57640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41120c0 con 0x7f7ca4101df0 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca41027a0 0x7f7ca4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35094/0 (socket says 172.21.15.28:35094) 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca93f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 0x7f7ca419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:17.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 -- 172.21.15.28:0/1034787390 learned_addr learned my addr 172.21.15.28:0/1034787390 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 -- 172.21.15.28:0/1034787390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca41027a0 msgr2=0x7f7ca4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca41027a0 0x7f7ca4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 -- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 msgr2=0x7f7ca419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 0x7f7ca419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 -- 172.21.15.28:0/1034787390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca419d3f0 con 0x7f7ca4101df0 2024-09-19T09:37:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca93f4640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 0x7f7ca419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca8bf3640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800e9f0 tx=0x7f7c9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800cd80 con 0x7f7ca4101df0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98010910 con 0x7f7ca4101df0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca419d6e0 con 0x7f7ca4101df0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.622+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98018990 con 0x7f7ca4101df0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.623+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca4074420 con 0x7f7ca4101df0 2024-09-19T09:37:17.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.624+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca41032d0 con 0x7f7ca4101df0 2024-09-19T09:37:17.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.625+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7c98010ac0 con 0x7f7ca4101df0 2024-09-19T09:37:17.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.626+0000 7f7ca1ffb640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 0x7f7c940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:17.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.626+0000 7f7ca3fff640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 0x7f7c940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:17.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.626+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f7c980cfff0 con 0x7f7ca4101df0 2024-09-19T09:37:17.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.626+0000 7f7ca3fff640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 0x7f7c940a5700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009230 tx=0x7f7c9003c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:17.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.630+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c9809c730 con 0x7f7ca4101df0 2024-09-19T09:37:17.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:17 smithi038 ceph-mon[35271]: pgmap v477: 97 pgs: 97 active+clean; 384 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 1.44k op/s 2024-09-19T09:37:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.852+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ca41021d0 con 0x7f7ca4101df0 2024-09-19T09:37:17.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.853+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1034787390 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11958 (secure 0 0 0) 0x7f7c980a0a30 con 0x7f7ca4101df0 2024-09-19T09:37:17.855 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:37:17.856 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":35,"btime":"2024-09-19T09:37:05:233386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:37:05.233357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:37:17.856 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-09-19T09:37:17.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 msgr2=0x7f7c940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:17.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 0x7f7c940a5700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009230 tx=0x7f7c9003c040 comp rx=0 tx=0).stop 2024-09-19T09:37:17.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 msgr2=0x7f7ca4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:17.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800e9f0 tx=0x7f7c9800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 shutdown_connections 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7c940a3240 0x7f7c940a5700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ca41027a0 0x7f7ca4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca4106f20 0x7f7ca419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 --2- 172.21.15.28:0/1034787390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca4101df0 0x7f7ca4198440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 >> 172.21.15.28:0/1034787390 conn(0x7f7ca40fd8f0 msgr2=0x7f7ca40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 shutdown_connections 2024-09-19T09:37:17.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:17.855+0000 7f7caae57640 1 -- 172.21.15.28:0/1034787390 wait complete. 2024-09-19T09:37:18.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:18 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1034787390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:18.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:18 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1034787390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:18.661 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id c9c0d8a --release --await' 2024-09-19T09:37:18.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id c9c0d8a --release --await 2024-09-19T09:37:18.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:18 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1034787390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- 172.21.15.28:0/1462265777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c101c30 msgr2=0x7ff55c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 --2- 172.21.15.28:0/1462265777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c101c30 0x7ff55c1020b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff550009900 tx=0x7ff55002a410 comp rx=0 tx=0).stop 2024-09-19T09:37:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- 172.21.15.28:0/1462265777 shutdown_connections 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 --2- 172.21.15.28:0/1462265777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c1025f0 0x7ff55c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 --2- 172.21.15.28:0/1462265777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c101c30 0x7ff55c1020b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 --2- 172.21.15.28:0/1462265777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c100a40 0x7ff55c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- 172.21.15.28:0/1462265777 >> 172.21.15.28:0/1462265777 conn(0x7ff55c0fc1c0 msgr2=0x7ff55c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- 172.21.15.28:0/1462265777 shutdown_connections 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- 172.21.15.28:0/1462265777 wait complete. 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 Processor -- start 2024-09-19T09:37:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.770+0000 7ff561731640 1 -- start start 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c101c30 0x7ff55c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c1025f0 0x7ff55c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c10c6a0 con 0x7ff55c100a40 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c10c520 con 0x7ff55c101c30 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff561731640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c10c820 con 0x7ff55c1025f0 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36062/0 (socket says 172.21.15.28:36062) 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 -- 172.21.15.28:0/3885335416 learned_addr learned my addr 172.21.15.28:0/3885335416 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c1025f0 0x7ff55c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c101c30 0x7ff55c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 -- 172.21.15.28:0/3885335416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c1025f0 msgr2=0x7ff55c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c1025f0 0x7ff55c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 -- 172.21.15.28:0/3885335416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c101c30 msgr2=0x7ff55c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c101c30 0x7ff55c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 -- 172.21.15.28:0/3885335416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c19faa0 con 0x7ff55c100a40 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff55affd640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d20 tx=0x7ff54c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.771+0000 7ff543fff640 1 -- 172.21.15.28:0/3885335416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c010630 con 0x7ff55c100a40 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.772+0000 7ff543fff640 1 -- 172.21.15.28:0/3885335416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c010cb0 con 0x7ff55c100a40 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.772+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c19fd90 con 0x7ff55c100a40 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.772+0000 7ff543fff640 1 -- 172.21.15.28:0/3885335416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c018de0 con 0x7ff55c100a40 2024-09-19T09:37:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.772+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c1a01f0 con 0x7ff55c100a40 2024-09-19T09:37:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 Processor -- start 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff543fff640 1 -- 172.21.15.28:0/3885335416 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff54c01f020 con 0x7ff55c100a40 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 -- start start 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d18832da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d18833370 0x559d18835830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559d187912f0 con 0x559d1882d9a0 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559d18739270 con 0x559d18833370 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff5644cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d1878d2b0 con 0x559d1882e2e0 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36074/0 (socket says 172.21.15.28:36074) 2024-09-19T09:37:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff55affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d18832da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff55b7fe640 1 -- 172.21.15.28:0/1958657120 learned_addr learned my addr 172.21.15.28:0/1958657120 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.774+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d18833370 0x559d18835830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 -- 172.21.15.28:0/1958657120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d18833370 msgr2=0x559d18835830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d18833370 0x559d18835830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 -- 172.21.15.28:0/1958657120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 msgr2=0x559d1882dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 -- 172.21.15.28:0/1958657120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d18738340 con 0x559d1882e2e0 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff543fff640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 0x7ff5440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55affd640 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d18832da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0cd8e0 tx=0x7ff54c0d6460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e38a1efb73e069f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 0x7ff5440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff543fff640 1 -- 172.21.15.28:0/3885335416 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff54c0cfde0 con 0x7ff55c100a40 2024-09-19T09:37:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff5417fa640 1 -- 172.21.15.28:0/1958657120 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0986d0 con 0x559d1882e2e0 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff5417fa640 1 -- 172.21.15.28:0/1958657120 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c09cb30 con 0x559d1882e2e0 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.775+0000 7ff5417fa640 1 -- 172.21.15.28:0/1958657120 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c09bd90 con 0x559d1882e2e0 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 -- 172.21.15.28:0/1958657120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 msgr2=0x559d18832da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 0x7ff5440a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff554012560 tx=0x7ff554005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d18832da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0cd8e0 tx=0x7ff54c0d6460 comp rx=0 tx=0).stop 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 -- 172.21.15.28:0/1958657120 shutdown_connections 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d18833370 0x559d18835830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d18832da0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1958657120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1882dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 -- 172.21.15.28:0/1958657120 >> 172.21.15.28:0/1958657120 conn(0x559d18829010 msgr2=0x559d18829410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 -- 172.21.15.28:0/1958657120 shutdown_connections 2024-09-19T09:37:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.776+0000 7ff5644cb740 1 -- 172.21.15.28:0/1958657120 wait complete. 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 -- start start 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 Processor -- start 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18838070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 0x559d1883f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559d188362e0 con 0x559d1882d9a0 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559d1871a290 con 0x559d1882e2e0 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d18836460 con 0x559d18833370 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff5644cb740 10 client.0 _ll_register_callbacks cb 0x559d1874b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36078/0 (socket says 172.21.15.28:36078) 2024-09-19T09:37:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55affd640 1 -- 172.21.15.28:0/1917952076 learned_addr learned my addr 172.21.15.28:0/1917952076 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 0x559d1883f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18838070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 -- 172.21.15.28:0/1917952076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 msgr2=0x559d18838070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18838070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 -- 172.21.15.28:0/1917952076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 msgr2=0x559d18837b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55b7fe640 1 -- 172.21.15.28:0/1917952076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d184a4210 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.777+0000 7ff55affd640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 0x559d1883f0b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff554012df0 tx=0x7ff554004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d482ea656bcd1c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554009830 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c09cb30 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff554018030 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff554005020 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 10 client.26104 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d18570620 con 0x559d18833370 2024-09-19T09:37:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c09cb30 con 0x559d18833370 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ff55401d620 con 0x559d18833370 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 10 client.26104 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 10 client.26104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.778+0000 7ff5644cb740 20 client.26104 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18843870 0x559d18845cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff5644cb740 4 client.26104 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d1879ea20 con 0x559d18843870 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff55affd640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18843870 0x559d18845cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff53eff5640 10 client.26104 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.779+0000 7ff55affd640 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18843870 0x559d18845cd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e74082ccdbb1d8d server_cookie=6e606e83406ffca in_seq=0 out_seq=0 2024-09-19T09:37:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff53eff5640 1 -- 172.21.15.28:0/1917952076 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d1879ea20 con 0x559d18843870 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff53eff5640 10 client.26104 handle_command_reply: tid=0 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 client.26104 shutdown 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 msgr2=0x559d1883f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 0x559d1883f0b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff554012df0 tx=0x7ff554004730 comp rx=0 tx=0).stop 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 shutdown_connections 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18838070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18843870 0x559d18845cd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18833370 0x559d1883f0b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff53eff5640 0 client.26104 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1917952076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d18837b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.781+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 >> 172.21.15.28:0/1917952076 conn(0x559d18829010 msgr2=0x559d1882a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.782+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 shutdown_connections 2024-09-19T09:37:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.782+0000 7ff5644cb740 1 -- 172.21.15.28:0/1917952076 wait complete. 2024-09-19T09:37:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.782+0000 7ff5644cb740 20 client.26104 trim_cache size 0 max 16384 2024-09-19T09:37:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 Processor -- start 2024-09-19T09:37:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 -- start start 2024-09-19T09:37:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1883efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18827ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1a8f90 con 0x559d1882d9a0 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1aab60 con 0x559d1882e2e0 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff5644cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1a9660 con 0x559d18829010 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff55a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1883efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff55affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff55affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35136/0 (socket says 172.21.15.28:35136) 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff55a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1883efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36094/0 (socket says 172.21.15.28:36094) 2024-09-19T09:37:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.797+0000 7ff55affd640 1 -- 172.21.15.28:0/14029560 learned_addr learned my addr 172.21.15.28:0/14029560 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18827ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 -- 172.21.15.28:0/14029560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 msgr2=0x559d18827ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18827ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 -- 172.21.15.28:0/14029560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 msgr2=0x559d1883efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1883efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 -- 172.21.15.28:0/14029560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d18837670 con 0x559d18829010 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff55affd640 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff54c098890 tx=0x7ff54c098cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4eb241a5f756a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff540ff9640 1 -- 172.21.15.28:0/14029560 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c010e60 con 0x559d18829010 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff540ff9640 1 -- 172.21.15.28:0/14029560 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c09cb30 con 0x559d18829010 2024-09-19T09:37:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.798+0000 7ff540ff9640 1 -- 172.21.15.28:0/14029560 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c09bf70 con 0x559d18829010 2024-09-19T09:37:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- 172.21.15.28:0/14029560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 msgr2=0x559d188279a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff54c098890 tx=0x7ff54c098cb0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- 172.21.15.28:0/14029560 shutdown_connections 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882e2e0 0x559d18827ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d1882d9a0 0x559d1883efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 --2- 172.21.15.28:0/14029560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d18829010 0x559d188279a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- 172.21.15.28:0/14029560 >> 172.21.15.28:0/14029560 conn(0x559d1882eda0 msgr2=0x559d18837b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- 172.21.15.28:0/14029560 shutdown_connections 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- 172.21.15.28:0/14029560 wait complete. 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 -- start start 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.799+0000 7ff5644cb740 1 Processor -- start 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882d9a0 0x559d1883cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d1883dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1aa490 con 0x559d18829010 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1a9d30 con 0x559d1882d9a0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1a6f70 con 0x559d1882e2e0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 10 client.0 _ll_register_callbacks cb 0x7ff554009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36108/0 (socket says 172.21.15.28:36108) 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 -- 172.21.15.28:0/1487870972 learned_addr learned my addr 172.21.15.28:0/1487870972 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55b7fe640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d1883dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882d9a0 0x559d1883cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 -- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 msgr2=0x559d1883dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d1883dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 -- 172.21.15.28:0/1487870972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882d9a0 msgr2=0x559d1883cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882d9a0 0x559d1883cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 -- 172.21.15.28:0/1487870972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d18835ce0 con 0x559d18829010 2024-09-19T09:37:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff55affd640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0a4b70 tx=0x7ff54c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4b6c513c3ccac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff53eff5640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c010e60 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7ff54c0184b0 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.800+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c09cb30 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff54c09c200 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 10 client.16560 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d188ae5b0 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c09cb30 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ff54c09c690 con 0x559d18829010 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 10 client.16560 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 20 client.16560 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18833370 0x559d188b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 4 client.16560 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:37:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d18798d70 con 0x559d18833370 2024-09-19T09:37:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.801+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18833370 0x559d188b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.802+0000 7ff53eff5640 10 client.16560 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:18.802+0000 7ff55a7fc640 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18833370 0x559d188b0b80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e3c0d75f85637e server_cookie=cb09fc6b4d1118af in_seq=0 out_seq=0 2024-09-19T09:37:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:19.058 INFO:tasks.workunit.client.0.smithi028.stdout: 114 2103 0 1874 0 2312 0 2024-09-19T09:37:19.346 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:19 smithi028 ceph-mon[30972]: pgmap v478: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 -- 172.21.15.28:0/3813433828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8101de0 msgr2=0x7f64b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 --2- 172.21.15.28:0/3813433828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8101de0 0x7f64b81021c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a802a410 comp rx=0 tx=0).stop 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 -- 172.21.15.28:0/3813433828 shutdown_connections 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 --2- 172.21.15.28:0/3813433828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8102790 0x7f64b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 --2- 172.21.15.28:0/3813433828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8106f10 0x7f64b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 --2- 172.21.15.28:0/3813433828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8101de0 0x7f64b81021c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 -- 172.21.15.28:0/3813433828 >> 172.21.15.28:0/3813433828 conn(0x7f64b80fd8c0 msgr2=0x7f64b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 -- 172.21.15.28:0/3813433828 shutdown_connections 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.437+0000 7f64be7dd640 1 -- 172.21.15.28:0/3813433828 wait complete. 2024-09-19T09:37:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 Processor -- start 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 -- start start 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 0x7f64b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8106f10 0x7f64b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111f30 con 0x7f64b8102790 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111db0 con 0x7f64b8106f10 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.438+0000 7f64be7dd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81120b0 con 0x7f64b8101de0 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 0x7f64b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35166/0 (socket says 172.21.15.28:35166) 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b7fff640 1 -- 172.21.15.28:0/2594407332 learned_addr learned my addr 172.21.15.28:0/2594407332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8106f10 0x7f64b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 -- 172.21.15.28:0/2594407332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 msgr2=0x7f64b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 -- 172.21.15.28:0/2594407332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8106f10 msgr2=0x7f64b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8106f10 0x7f64b8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 -- 172.21.15.28:0/2594407332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b81a6990 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 0x7f64b8074a30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d6d0 tx=0x7f64ac00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b57fa640 1 -- 172.21.15.28:0/2594407332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac017070 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b81a6c80 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b57fa640 1 -- 172.21.15.28:0/2594407332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ac009820 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64b57fa640 1 -- 172.21.15.28:0/2594407332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac004cd0 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.439+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a71c0 con 0x7f64b8102790 2024-09-19T09:37:19.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64b57fa640 1 -- 172.21.15.28:0/2594407332 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f64ac0099d0 con 0x7f64b8102790 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 Processor -- start 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 -- start start 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 0x55ae07387d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae07388240 0x55ae0738a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae072e2620 con 0x55ae07388240 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae07289e50 con 0x55ae0737d980 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae072de5e0 con 0x55ae0737d060 2024-09-19T09:37:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64bcd7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae07388240 0x55ae0738a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64bcd7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35180/0 (socket says 172.21.15.28:35180) 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64bcd7a640 1 -- 172.21.15.28:0/2474391089 learned_addr learned my addr 172.21.15.28:0/2474391089 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.442+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 0x55ae07387d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 -- 172.21.15.28:0/2474391089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 msgr2=0x55ae07387d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 0x55ae07387d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 -- 172.21.15.28:0/2474391089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae07388240 msgr2=0x55ae0738a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae07388240 0x55ae0738a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 -- 172.21.15.28:0/2474391089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae0727ad40 con 0x55ae0737d060 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64b57fa640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 0x7f64a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 0x55ae07387d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 0x7f64a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f50 tx=0x7f64b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e6521fde9a80a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f64b57fa640 1 -- 172.21.15.28:0/2594407332 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f64ac0cf3d0 con 0x7f64b8102790 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f649e7fc640 1 -- 172.21.15.28:0/2474391089 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004b20 con 0x55ae0737d060 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f649e7fc640 1 -- 172.21.15.28:0/2474391089 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0004cc0 con 0x55ae0737d060 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.443+0000 7f649e7fc640 1 -- 172.21.15.28:0/2474391089 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0005020 con 0x55ae0737d060 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2474391089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 msgr2=0x55ae0737d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 0x7f64a00a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55ae0737a450 tx=0x7f64a8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f50 tx=0x7f64b00118d0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2474391089 shutdown_connections 2024-09-19T09:37:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae0737d980 0x55ae07387d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae07388240 0x55ae0738a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2474391089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae0737d440 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2474391089 >> 172.21.15.28:0/2474391089 conn(0x55ae073795f0 msgr2=0x55ae073799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2474391089 shutdown_connections 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2474391089 wait complete. 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.444+0000 7f64c0bb6740 1 -- start start 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 Processor -- start 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07385140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae07385bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae0738ae10 con 0x55ae0737d980 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae0726af10 con 0x55ae07388240 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae0738af90 con 0x55ae0737d060 2024-09-19T09:37:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64c0bb6740 10 client.0 _ll_register_callbacks cb 0x55ae0728edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36138/0 (socket says 172.21.15.28:36138) 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 -- 172.21.15.28:0/936617628 learned_addr learned my addr 172.21.15.28:0/936617628 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07385140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae07385bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 -- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 msgr2=0x55ae07385140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07385140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 -- 172.21.15.28:0/936617628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 msgr2=0x55ae07385bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae07385bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.445+0000 7f64b77fe640 1 -- 172.21.15.28:0/936617628 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae06ff1580 con 0x55ae0737d980 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64b77fe640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009610 tx=0x7f64ac097d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc3d8f1dea242bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64977fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac020070 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f64ac098140 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0004cc0 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64ac09b430 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64c0bb6740 10 client.16572 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae070bb960 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.446+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0004cc0 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f64ac09af20 con 0x55ae0737d980 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 10 client.16572 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 10 client.16572 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 20 client.16572 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae07397f40 0x55ae0739a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 4 client.16572 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae072efb90 con 0x55ae07397f40 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64b7fff640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae07397f40 0x55ae0739a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64977fe640 10 client.16572 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.447+0000 7f64b7fff640 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae07397f40 0x55ae0739a330 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ff40c8e1f26114 server_cookie=b01f9efa1780a41d in_seq=0 out_seq=0 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64977fe640 1 -- 172.21.15.28:0/936617628 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae072efb90 con 0x55ae07397f40 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64977fe640 10 client.16572 handle_command_reply: tid=0 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 client.16572 shutdown 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 msgr2=0x55ae07385680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009610 tx=0x7f64ac097d70 comp rx=0 tx=0).stop 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 shutdown_connections 2024-09-19T09:37:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae07385bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae07397f40 0x55ae0739a330 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae07385680 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/936617628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07385140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 >> 172.21.15.28:0/936617628 conn(0x55ae073795f0 msgr2=0x55ae07378750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64977fe640 0 client.16572 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 shutdown_connections 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 1 -- 172.21.15.28:0/936617628 wait complete. 2024-09-19T09:37:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.449+0000 7f64c0bb6740 20 client.16572 trim_cache size 0 max 16384 2024-09-19T09:37:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.469+0000 7f64c0bb6740 1 Processor -- start 2024-09-19T09:37:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.469+0000 7f64c0bb6740 1 -- start start 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.469+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07378640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.469+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.469+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae073a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81af6c0 con 0x55ae0737d980 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b1290 con 0x55ae07388240 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81afd90 con 0x55ae0737d060 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36148/0 (socket says 172.21.15.28:36148) 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07378640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b77fe640 1 -- 172.21.15.28:0/573011414 learned_addr learned my addr 172.21.15.28:0/573011414 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae073a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 -- 172.21.15.28:0/573011414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 msgr2=0x55ae073a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae073a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 -- 172.21.15.28:0/573011414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 msgr2=0x55ae0739fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 -- 172.21.15.28:0/573011414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae07383690 con 0x55ae0737d060 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b77fe640 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f64b7fff640 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07378640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009230 tx=0x7f64a803e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c620fe2f279ee24e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.470+0000 7f649dffb640 1 -- 172.21.15.28:0/573011414 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8035040 con 0x55ae0737d060 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f649dffb640 1 -- 172.21.15.28:0/573011414 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a8056020 con 0x55ae0737d060 2024-09-19T09:37:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f649dffb640 1 -- 172.21.15.28:0/573011414 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004c90 con 0x55ae0737d060 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 -- 172.21.15.28:0/573011414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 msgr2=0x55ae07378640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07378640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009230 tx=0x7f64a803e070 comp rx=0 tx=0).stop 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 -- 172.21.15.28:0/573011414 shutdown_connections 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae073a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0739fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/573011414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07378640 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 -- 172.21.15.28:0/573011414 >> 172.21.15.28:0/573011414 conn(0x55ae073795f0 msgr2=0x55ae07377f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 -- 172.21.15.28:0/573011414 shutdown_connections 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.471+0000 7f64c0bb6740 1 -- 172.21.15.28:0/573011414 wait complete. 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 -- start start 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 Processor -- start 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0738d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae0738d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b0bc0 con 0x55ae0737d980 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b0460 con 0x55ae07388240 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.472+0000 7f64c0bb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ad6c0 con 0x55ae0737d060 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64c0bb6740 10 client.0 _ll_register_callbacks cb 0x7f64ac0057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0738d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35214/0 (socket says 172.21.15.28:35214) 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 -- 172.21.15.28:0/2832262425 learned_addr learned my addr 172.21.15.28:0/2832262425 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae0738d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 -- 172.21.15.28:0/2832262425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 msgr2=0x55ae0738d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae0738d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 -- 172.21.15.28:0/2832262425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 msgr2=0x55ae0738d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0738d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 -- 172.21.15.28:0/2832262425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae0738de10 con 0x55ae0737d060 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0738d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64bcd7a640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae0738d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64b7fff640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009a20 tx=0x7f64a8031950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d8b46a16f822b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64977fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8062020 con 0x55ae0737d060 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f64a8004a10 con 0x55ae0737d060 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.473+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a8056020 con 0x55ae0737d060 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64a8004c10 con 0x55ae0737d060 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 10 client.26128 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:19.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae07403af0 con 0x55ae0737d060 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a8056020 con 0x55ae0737d060 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f64a8003a60 con 0x55ae0737d060 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 10 client.26128 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 10 client.26128 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 20 client.26128 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae074041b0 0x55ae074065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 4 client.26128 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9c0d8a", "release": true, "await": true}] 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9c0d8a", "release": true, "await": true}) -- 0x55ae072ea0a0 con 0x55ae074041b0 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.474+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae074041b0 0x55ae074065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.475+0000 7f64977fe640 10 client.26128 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:19.475+0000 7f64b77fe640 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae074041b0 0x55ae074065d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45607ed7eece8033 server_cookie=7646df9474e75ffe in_seq=0 out_seq=0 2024-09-19T09:37:19.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:19 smithi078 ceph-mon[35619]: pgmap v478: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-09-19T09:37:19.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:19 smithi038 ceph-mon[35271]: pgmap v478: 97 pgs: 97 active+clean; 388 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-09-19T09:37:20.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64977fe640 1 -- 172.21.15.28:0/2832262425 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ae072ea0a0 con 0x55ae074041b0 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64977fe640 10 client.26128 handle_command_reply: tid=0 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 client.26128 shutdown 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 msgr2=0x55ae07390840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009a20 tx=0x7f64a8031950 comp rx=0 tx=0).stop 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 shutdown_connections 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae07388240 0x55ae0738d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ae074041b0 0x55ae074065d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ae0737d980 0x55ae0738d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 --2- 172.21.15.28:0/2832262425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae0737d060 0x55ae07390840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64977fe640 0 client.26128 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:20.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.399+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 >> 172.21.15.28:0/2832262425 conn(0x55ae073795f0 msgr2=0x55ae07385920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 shutdown_connections 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64c0bb6740 1 -- 172.21.15.28:0/2832262425 wait complete. 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64c0bb6740 20 client.26128 trim_cache size 0 max 16384 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 35, 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 20, 2024-09-19T09:37:20.403 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "c9c0d8a": { 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 20, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 149.8, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:37:20.404 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:20.405 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 msgr2=0x7f64a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 0x7f64a00a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55ae0737a450 tx=0x7f64a8005a20 comp rx=0 tx=0).stop 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 msgr2=0x7f64b8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.400+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 0x7f64b8074a30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d6d0 tx=0x7f64ac00db90 comp rx=0 tx=0).stop 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 shutdown_connections 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f64a00a3240 0x7f64a00a5700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64b8106f10 0x7f64b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64b8102790 0x7f64b8074a30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 --2- 172.21.15.28:0/2594407332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8101de0 0x7f64b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 >> 172.21.15.28:0/2594407332 conn(0x7f64b80fd8c0 msgr2=0x7f64b80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:20.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 shutdown_connections 2024-09-19T09:37:20.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:20.401+0000 7f64be7dd640 1 -- 172.21.15.28:0/2594407332 wait complete. 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.928+0000 7ff53eff5640 1 -- 172.21.15.28:0/1487870972 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d18798d70 con 0x559d18833370 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.928+0000 7ff53eff5640 10 client.16560 handle_command_reply: tid=0 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.928+0000 7ff5644cb740 1 client.16560 shutdown 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 msgr2=0x559d1883c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0a4b70 tx=0x7ff54c005e30 comp rx=0 tx=0).stop 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 shutdown_connections 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d1882d9a0 0x559d1883cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x559d18833370 0x559d188b0b80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d1882e2e0 0x559d1883dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 --2- 172.21.15.28:0/1487870972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559d18829010 0x559d1883c8a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 >> 172.21.15.28:0/1487870972 conn(0x559d1882eda0 msgr2=0x559d188281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff53eff5640 0 client.16560 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 shutdown_connections 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 1 -- 172.21.15.28:0/1487870972 wait complete. 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.929+0000 7ff5644cb740 20 client.16560 trim_cache size 0 max 16384 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:37:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:37:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:37:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:37:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 msgr2=0x7ff5440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 0x7ff5440a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff554012560 tx=0x7ff554005f90 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 msgr2=0x7ff55c06a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009d20 tx=0x7ff54c00c6a0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 shutdown_connections 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff5440a3240 0x7ff5440a5700 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55c101c30 0x7ff55c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff55c1025f0 0x7ff55c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 --2- 172.21.15.28:0/3885335416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff55c100a40 0x7ff55c06a5c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 >> 172.21.15.28:0/3885335416 conn(0x7ff55c0fc1c0 msgr2=0x7ff55c1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 shutdown_connections 2024-09-19T09:37:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:20.930+0000 7ff561731640 1 -- 172.21.15.28:0/3885335416 wait complete. 2024-09-19T09:37:21.158 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c9c0d8a', total seconds elapsed: 34.6 2024-09-19T09:37:21.158 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T09:37:21.158 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:37:21.430 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:21.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:21 smithi028 ceph-mon[30972]: pgmap v479: 97 pgs: 97 active+clean; 393 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 782 op/s 2024-09-19T09:37:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:21 smithi078 ceph-mon[35619]: pgmap v479: 97 pgs: 97 active+clean; 393 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 782 op/s 2024-09-19T09:37:21.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:21 smithi038 ceph-mon[35271]: pgmap v479: 97 pgs: 97 active+clean; 393 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 782 op/s 2024-09-19T09:37:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 -- 172.21.15.28:0/1072109135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464101420 msgr2=0x7fa46410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/1072109135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464101420 0x7fa46410d950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009cd0 tx=0x7fa45c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 -- 172.21.15.28:0/1072109135 shutdown_connections 2024-09-19T09:37:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/1072109135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/1072109135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464101420 0x7fa46410d950 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/1072109135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464106a80 0x7fa464106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 -- 172.21.15.28:0/1072109135 >> 172.21.15.28:0/1072109135 conn(0x7fa4640fc7a0 msgr2=0x7fa4640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 -- 172.21.15.28:0/1072109135 shutdown_connections 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.192+0000 7fa46bbbc640 1 -- 172.21.15.28:0/1072109135 wait complete. 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 Processor -- start 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 -- start start 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 0x7fa464197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 0x7fa464197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa464110e80 con 0x7fa464106a80 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa464110d00 con 0x7fa464100a80 2024-09-19T09:37:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46bbbc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa464111000 con 0x7fa464101420 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa46a159640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 0x7fa464197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa469958640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa469958640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:55684/0 (socket says 172.21.15.28:55684) 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa469157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 0x7fa464197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.193+0000 7fa469958640 1 -- 172.21.15.28:0/2390358483 learned_addr learned my addr 172.21.15.28:0/2390358483 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 -- 172.21.15.28:0/2390358483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 msgr2=0x7fa464197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 0x7fa464197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 -- 172.21.15.28:0/2390358483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 msgr2=0x7fa464197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 -- 172.21.15.28:0/2390358483 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4641a68b0 con 0x7fa464106a80 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa469157640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 0x7fa464197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa469958640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46a159640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 0x7fa464197ac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa45c02ae00 tx=0x7fa45c031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c003d80 con 0x7fa464106a80 2024-09-19T09:37:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa45c004520 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c030d00 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4641a6b40 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.194+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4641a7080 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.195+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa464100ee0 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.196+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa45c0046d0 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.197+0000 7fa45affd640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 0x7fa4500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.197+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fa45c02e030 con 0x7fa464106a80 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.197+0000 7fa469958640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 0x7fa4500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.197+0000 7fa469958640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 0x7fa4500a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa460011490 tx=0x7fa460005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.201+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa45c04b060 con 0x7fa464106a80 2024-09-19T09:37:22.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.425+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4641a73c0 con 0x7fa464106a80 2024-09-19T09:37:22.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.426+0000 7fa45affd640 1 -- 172.21.15.28:0/2390358483 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11958 (secure 0 0 0) 0x7fa45c0bdbc0 con 0x7fa464106a80 2024-09-19T09:37:22.427 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:37:22.427 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":35,"btime":"2024-09-19T09:37:05:233386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:37:05.233357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:37:22.428 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-09-19T09:37:22.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 msgr2=0x7fa4500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:22.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 0x7fa4500a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa460011490 tx=0x7fa460005f90 comp rx=0 tx=0).stop 2024-09-19T09:37:22.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 msgr2=0x7fa464197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:22.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 0x7fa464197ac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa45c02ae00 tx=0x7fa45c031df0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 shutdown_connections 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa4500a3240 0x7fa4500a5700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa464106a80 0x7fa464197ac0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa464101420 0x7fa464197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 --2- 172.21.15.28:0/2390358483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa464100a80 0x7fa464197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 >> 172.21.15.28:0/2390358483 conn(0x7fa4640fc7a0 msgr2=0x7fa4640fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 shutdown_connections 2024-09-19T09:37:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:22.428+0000 7fa46bbbc640 1 -- 172.21.15.28:0/2390358483 wait complete. 2024-09-19T09:37:23.360 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T09:37:23.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T09:37:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:23 smithi078 ceph-mon[35619]: pgmap v480: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2024-09-19T09:37:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:23 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2390358483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:23.636 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:23.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:23 smithi038 ceph-mon[35271]: pgmap v480: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2024-09-19T09:37:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:23 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2390358483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:23.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:23 smithi028 ceph-mon[30972]: pgmap v480: 97 pgs: 97 active+clean; 385 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 331 op/s 2024-09-19T09:37:23.762 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:23 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2390358483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 -- 172.21.15.28:0/3476337939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec1082e0 msgr2=0x7fb8ec10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/3476337939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec10ec50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009cd0 tx=0x7fb8e402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 -- 172.21.15.28:0/3476337939 shutdown_connections 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/3476337939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec10ec50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/3476337939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec103760 0x7fb8ec107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/3476337939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 0x7fb8ec103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 -- 172.21.15.28:0/3476337939 >> 172.21.15.28:0/3476337939 conn(0x7fb8ec0fc7d0 msgr2=0x7fb8ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 -- 172.21.15.28:0/3476337939 shutdown_connections 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.167+0000 7fb8f3cce640 1 -- 172.21.15.28:0/3476337939 wait complete. 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 Processor -- start 2024-09-19T09:37:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 -- start start 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 0x7fb8ec1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1120b0 con 0x7fb8ec103760 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec111f30 con 0x7fb8ec102db0 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f3cce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec112230 con 0x7fb8ec1082e0 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56020/0 (socket says 172.21.15.28:56020) 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.168+0000 7fb8f1269640 1 -- 172.21.15.28:0/2507564073 learned_addr learned my addr 172.21.15.28:0/2507564073 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 0x7fb8ec1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 -- 172.21.15.28:0/2507564073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec1082e0 msgr2=0x7fb8ec19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 -- 172.21.15.28:0/2507564073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 msgr2=0x7fb8ec1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 0x7fb8ec1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 -- 172.21.15.28:0/2507564073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec19acd0 con 0x7fb8ec103760 2024-09-19T09:37:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8e2ffd640 1 -- 172.21.15.28:0/2507564073 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00ccc0 con 0x7fb8ec103760 2024-09-19T09:37:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8e2ffd640 1 -- 172.21.15.28:0/2507564073 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8dc005500 con 0x7fb8ec103760 2024-09-19T09:37:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec19afc0 con 0x7fb8ec103760 2024-09-19T09:37:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8e2ffd640 1 -- 172.21.15.28:0/2507564073 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc010690 con 0x7fb8ec103760 2024-09-19T09:37:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.169+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1a72b0 con 0x7fb8ec103760 2024-09-19T09:37:24.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.172+0000 7fb8e2ffd640 1 -- 172.21.15.28:0/2507564073 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb8dc004e80 con 0x7fb8ec103760 2024-09-19T09:37:24.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.172+0000 7fb8f60a7740 1 Processor -- start 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 -- start start 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 0x556ac93f6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ac93512a0 con 0x556ac93f6940 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ac92f8b10 con 0x556ac93ec080 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ac934d260 con 0x556ac93eb760 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f226b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f226b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45564/0 (socket says 172.21.15.28:45564) 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f1a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 0x556ac93f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f226b640 1 -- 172.21.15.28:0/2078102960 learned_addr learned my addr 172.21.15.28:0/2078102960 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.173+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 -- 172.21.15.28:0/2078102960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 msgr2=0x556ac93f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8e2ffd640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 0x7fb8d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 0x556ac93f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 -- 172.21.15.28:0/2078102960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 msgr2=0x556ac93f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 -- 172.21.15.28:0/2078102960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac90c2b90 con 0x556ac93eb760 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 0x556ac93f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8e2ffd640 1 -- 172.21.15.28:0/2507564073 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb8dc0cfbb0 con 0x7fb8ec103760 2024-09-19T09:37:24.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 0x7fb8d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009cd0 tx=0x7fb8e4009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e7f77d5eb63e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8d7fff640 1 -- 172.21.15.28:0/2078102960 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4003e40 con 0x556ac93eb760 2024-09-19T09:37:24.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 0x7fb8d80a5780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40318b0 tx=0x7fb8e404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8d7fff640 1 -- 172.21.15.28:0/2078102960 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4036070 con 0x556ac93eb760 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.174+0000 7fb8d7fff640 1 -- 172.21.15.28:0/2078102960 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4030c40 con 0x556ac93eb760 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2078102960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 msgr2=0x556ac93ebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009cd0 tx=0x7fb8e4009a20 comp rx=0 tx=0).stop 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2078102960 shutdown_connections 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93ec080 0x556ac93f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2078102960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93ebb40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2078102960 >> 172.21.15.28:0/2078102960 conn(0x556ac93e7cf0 msgr2=0x556ac93e80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2078102960 shutdown_connections 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.175+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2078102960 wait complete. 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 -- start start 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 Processor -- start 2024-09-19T09:37:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93f6940 0x556ac93f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ac93f9510 con 0x556ac93ec080 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556ac92d96e0 con 0x556ac93eb760 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ac93f9690 con 0x556ac93f6940 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f60a7740 10 client.0 _ll_register_callbacks cb 0x556ac92fda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f1a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.176+0000 7fb8f226b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93f6940 0x556ac93f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56034/0 (socket says 172.21.15.28:56034) 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 -- 172.21.15.28:0/2815721332 learned_addr learned my addr 172.21.15.28:0/2815721332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:49032/0 (socket says 172.21.15.28:49032) 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 -- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93f6940 msgr2=0x556ac93f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93f6940 0x556ac93f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 -- 172.21.15.28:0/2815721332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 msgr2=0x556ac93f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 -- 172.21.15.28:0/2815721332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac9402a80 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f1269640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c6a0 tx=0x7fb8dc0d3280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7b5aee8f33149b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc09bb60 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb8dc005170 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e4036070 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8dc00ce60 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f60a7740 10 client.16596 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.177+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ac94054c0 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4036070 con 0x556ac93ec080 2024-09-19T09:37:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb8dc09c460 con 0x556ac93ec080 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 10 client.16596 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 10 client.16596 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 20 client.16596 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9406980 0x556ac9408dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 4 client.16596 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ac935e9d0 con 0x556ac9406980 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9406980 0x556ac9408dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.178+0000 7fb8d57fa640 10 client.16596 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.179+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9406980 0x556ac9408dd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3c02fe5ce6ac3 server_cookie=1e8d12b9055c7577 in_seq=0 out_seq=0 2024-09-19T09:37:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8d57fa640 1 -- 172.21.15.28:0/2815721332 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ac935e9d0 con 0x556ac9406980 2024-09-19T09:37:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8d57fa640 10 client.16596 handle_command_reply: tid=0 2024-09-19T09:37:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 client.16596 shutdown 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 msgr2=0x556ac93f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c6a0 tx=0x7fb8dc0d3280 comp rx=0 tx=0).stop 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 shutdown_connections 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9406980 0x556ac9408dd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93f6940 0x556ac93f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93f3df0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2815721332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac93f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8d57fa640 0 client.16596 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.180+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 >> 172.21.15.28:0/2815721332 conn(0x556ac93e7cf0 msgr2=0x556ac93e6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.181+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 shutdown_connections 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.181+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2815721332 wait complete. 2024-09-19T09:37:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.181+0000 7fb8f60a7740 20 client.16596 trim_cache size 0 max 16384 2024-09-19T09:37:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.198+0000 7fb8f60a7740 1 Processor -- start 2024-09-19T09:37:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.198+0000 7fb8f60a7740 1 -- start start 2024-09-19T09:37:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac940e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93ec080 0x556ac940ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1af6d0 con 0x556ac93f6940 2024-09-19T09:37:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b12a0 con 0x556ac93eb760 2024-09-19T09:37:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1afda0 con 0x556ac93ec080 2024-09-19T09:37:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93ec080 0x556ac940ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56046/0 (socket says 172.21.15.28:56046) 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 -- 172.21.15.28:0/2025524387 learned_addr learned my addr 172.21.15.28:0/2025524387 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac940e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 -- 172.21.15.28:0/2025524387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93ec080 msgr2=0x556ac940ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93ec080 0x556ac940ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 -- 172.21.15.28:0/2025524387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 msgr2=0x556ac940e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac940e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 -- 172.21.15.28:0/2025524387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac9409450 con 0x556ac93f6940 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8f226b640 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4004980 tx=0x7fb8e4052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe19306c573d6a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8d77fe640 1 -- 172.21.15.28:0/2025524387 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40305a0 con 0x556ac93f6940 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.199+0000 7fb8d77fe640 1 -- 172.21.15.28:0/2025524387 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4036040 con 0x556ac93f6940 2024-09-19T09:37:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8d77fe640 1 -- 172.21.15.28:0/2025524387 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4051b40 con 0x556ac93f6940 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2025524387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 msgr2=0x556ac93f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4004980 tx=0x7fb8e4052000 comp rx=0 tx=0).stop 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2025524387 shutdown_connections 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93f6940 0x556ac93f0b10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93ec080 0x556ac940ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/2025524387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93eb760 0x556ac940e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2025524387 >> 172.21.15.28:0/2025524387 conn(0x556ac93e7cf0 msgr2=0x556ac93f5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2025524387 shutdown_connections 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.200+0000 7fb8f60a7740 1 -- 172.21.15.28:0/2025524387 wait complete. 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 -- start start 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 Processor -- start 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93f6940 0x556ac93fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b0bd0 con 0x556ac93ec080 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b0470 con 0x556ac93f6940 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1ad6b0 con 0x556ac93eb760 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f60a7740 10 client.0 _ll_register_callbacks cb 0x7fb8dc0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f1a6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f1a6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:45586/0 (socket says 172.21.15.28:45586) 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f1a6a640 1 -- 172.21.15.28:0/1067254175 learned_addr learned my addr 172.21.15.28:0/1067254175 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f1269640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56048/0 (socket says 172.21.15.28:56048) 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.201+0000 7fb8f226b640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93f6940 0x556ac93fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 -- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 msgr2=0x556ac93fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 -- 172.21.15.28:0/1067254175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93f6940 msgr2=0x556ac93fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93f6940 0x556ac93fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 -- 172.21.15.28:0/1067254175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac93e3260 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f1269640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x556ac93f4e60 tx=0x7fb8dc0a4fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf04fd78c52b91a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc09bb60 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb8dc0a4be0 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e4036040 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8dc0b51c0 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f60a7740 10 client.16602 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.202+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ac93eb590 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e4036040 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb8dc0a0120 con 0x556ac93ec080 2024-09-19T09:37:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 10 client.16602 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 10 client.16602 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 20 client.16602 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9472490 0x556ac94748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 4 client.16602 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x556ac9358d20 con 0x556ac9472490 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9472490 0x556ac94748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8f1a6a640 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9472490 0x556ac94748b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb5274f957c5fbc server_cookie=c7ce6c6ce32a7bf3 in_seq=0 out_seq=0 2024-09-19T09:37:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:24.203+0000 7fb8d57fa640 10 client.16602 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:25.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:25 smithi078 ceph-mon[35619]: pgmap v481: 97 pgs: 97 active+clean; 378 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2024-09-19T09:37:25.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:25 smithi038 ceph-mon[35271]: pgmap v481: 97 pgs: 97 active+clean; 378 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2024-09-19T09:37:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:25 smithi028 ceph-mon[30972]: pgmap v481: 97 pgs: 97 active+clean; 378 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2024-09-19T09:37:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:27 smithi078 ceph-mon[35619]: pgmap v482: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 276 op/s 2024-09-19T09:37:27.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:27 smithi038 ceph-mon[35271]: pgmap v482: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 276 op/s 2024-09-19T09:37:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:27 smithi028 ceph-mon[30972]: pgmap v482: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 276 op/s 2024-09-19T09:37:29.058 INFO:tasks.workunit.client.0.smithi028.stdout: 115 5864 60 4579 42 3688 119 2024-09-19T09:37:29.566 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:29 smithi028 ceph-mon[30972]: pgmap v483: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-19T09:37:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:29 smithi078 ceph-mon[35619]: pgmap v483: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-19T09:37:29.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:29 smithi038 ceph-mon[35271]: pgmap v483: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-09-19T09:37:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:31 smithi078 ceph-mon[35619]: pgmap v484: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-19T09:37:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:31 smithi038 ceph-mon[35271]: pgmap v484: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-19T09:37:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:31 smithi028 ceph-mon[30972]: pgmap v484: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-19T09:37:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:33 smithi078 ceph-mon[35619]: pgmap v485: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 905 KiB/s wr, 355 op/s 2024-09-19T09:37:33.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:33 smithi038 ceph-mon[35271]: pgmap v485: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 905 KiB/s wr, 355 op/s 2024-09-19T09:37:33.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:33 smithi028 ceph-mon[30972]: pgmap v485: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 905 KiB/s wr, 355 op/s 2024-09-19T09:37:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- 172.21.15.28:0/429061096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 msgr2=0x7f0050101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 --2- 172.21.15.28:0/429061096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f0050101b70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0048009ca0 tx=0x7f004802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- 172.21.15.28:0/429061096 shutdown_connections 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 --2- 172.21.15.28:0/429061096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f0050101b70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 --2- 172.21.15.28:0/429061096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f0050101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 --2- 172.21.15.28:0/429061096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f00501007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- 172.21.15.28:0/429061096 >> 172.21.15.28:0/429061096 conn(0x7f00500fc1f0 msgr2=0x7f00500fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- 172.21.15.28:0/429061096 shutdown_connections 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- 172.21.15.28:0/429061096 wait complete. 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 Processor -- start 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.031+0000 7f00574ed640 1 -- start start 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f005019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f005019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f005010f9c0 con 0x7f00501016c0 2024-09-19T09:37:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f005010f840 con 0x7f00501003e0 2024-09-19T09:37:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f005010fb40 con 0x7f0050100d20 2024-09-19T09:37:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0054a88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f005019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47034/0 (socket says 172.21.15.28:47034) 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 -- 172.21.15.28:0/2471328807 learned_addr learned my addr 172.21.15.28:0/2471328807 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055289640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f005019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 -- 172.21.15.28:0/2471328807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 msgr2=0x7f005019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f005019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 -- 172.21.15.28:0/2471328807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 msgr2=0x7f005019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f005019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 -- 172.21.15.28:0/2471328807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00501a1c70 con 0x7f00501016c0 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0054a88640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f005019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055289640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f005019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f0055a8a640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0048005e30 tx=0x7f0048004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00467fc640 1 -- 172.21.15.28:0/2471328807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048003b70 con 0x7f00501016c0 2024-09-19T09:37:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00467fc640 1 -- 172.21.15.28:0/2471328807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048003d10 con 0x7f00501016c0 2024-09-19T09:37:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.032+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00501a1f00 con 0x7f00501016c0 2024-09-19T09:37:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.033+0000 7f00467fc640 1 -- 172.21.15.28:0/2471328807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004802fad0 con 0x7f00501016c0 2024-09-19T09:37:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.033+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501a2410 con 0x7f00501016c0 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 Processor -- start 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f00467fc640 1 -- 172.21.15.28:0/2471328807 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f00480045c0 con 0x7f00501016c0 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 -- start start 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d12e0 0x56297d2d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56297d2342f0 con 0x56297d2d6370 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56297d1dc270 con 0x56297d2d12e0 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f005a288740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56297d2302b0 con 0x56297d2d09a0 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47036/0 (socket says 172.21.15.28:47036) 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f0054a88640 1 -- 172.21.15.28:0/2896625271 learned_addr learned my addr 172.21.15.28:0/2896625271 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f0054a88640 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.036+0000 7f0055289640 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d12e0 0x56297d2d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 -- 172.21.15.28:0/2896625271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 msgr2=0x56297d2d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 -- 172.21.15.28:0/2896625271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d12e0 msgr2=0x56297d2d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d12e0 0x56297d2d5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 -- 172.21.15.28:0/2896625271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297d1db340 con 0x56297d2d6370 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f00467fc640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 0x7f003c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0054a88640 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0040011be0 tx=0x7f0040009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655055eb6531c718 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f0055289640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 0x7f003c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f00467fc640 1 -- 172.21.15.28:0/2471328807 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f00480f0ac0 con 0x7f00501016c0 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f003b7fe640 1 -- 172.21.15.28:0/2896625271 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000fc50 con 0x56297d2d6370 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f003b7fe640 1 -- 172.21.15.28:0/2896625271 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004000fdf0 con 0x56297d2d6370 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.037+0000 7f003b7fe640 1 -- 172.21.15.28:0/2896625271 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000e6a0 con 0x56297d2d6370 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- 172.21.15.28:0/2896625271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 msgr2=0x56297d2d8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f0055289640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 0x7f003c0a5780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56297d2d1d10 tx=0x7f004c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0040011be0 tx=0x7f0040009ba0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- 172.21.15.28:0/2896625271 shutdown_connections 2024-09-19T09:37:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d12e0 0x56297d2d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d6370 0x56297d2d8830 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 --2- 172.21.15.28:0/2896625271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- 172.21.15.28:0/2896625271 >> 172.21.15.28:0/2896625271 conn(0x56297d2cc010 msgr2=0x56297d2cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- 172.21.15.28:0/2896625271 shutdown_connections 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- 172.21.15.28:0/2896625271 wait complete. 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 -- start start 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.038+0000 7f005a288740 1 Processor -- start 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d6370 0x56297d2e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56297d2d9340 con 0x56297d2d12e0 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56297d1bd290 con 0x56297d2d6370 2024-09-19T09:37:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56297d2d94c0 con 0x56297d2d09a0 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f005a288740 10 client.0 _ll_register_callbacks cb 0x56297d1ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47044/0 (socket says 172.21.15.28:47044) 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 -- 172.21.15.28:0/4091846422 learned_addr learned my addr 172.21.15.28:0/4091846422 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0055a8a640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d6370 0x56297d2e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0055289640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 -- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 msgr2=0x56297d2dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2dac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 -- 172.21.15.28:0/4091846422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d6370 msgr2=0x56297d2e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d6370 0x56297d2e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 -- 172.21.15.28:0/4091846422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297cf47210 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.039+0000 7f0054a88640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f004001f8a0 tx=0x7f004001f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3af60420ef64e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000fc20 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f0040005030 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004000fdf0 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0040005250 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 10 client.16614 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56297d013620 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004000fdf0 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0040026070 con 0x56297d2d12e0 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 10 client.16614 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 20 client.16614 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d2e6740 0x56297d2e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 4 client.16614 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.040+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297d241a20 con 0x56297d2e6740 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.041+0000 7f0055289640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d2e6740 0x56297d2e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.041+0000 7f0055289640 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d2e6740 0x56297d2e8b60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac4d366b5159d8b server_cookie=6e9cdbb93dc8e82e in_seq=0 out_seq=0 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.041+0000 7f0038ff9640 10 client.16614 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f0038ff9640 1 -- 172.21.15.28:0/4091846422 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56297d241a20 con 0x56297d2e6740 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f0038ff9640 10 client.16614 handle_command_reply: tid=0 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 client.16614 shutdown 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 msgr2=0x56297d2db1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f004001f8a0 tx=0x7f004001f8d0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 shutdown_connections 2024-09-19T09:37:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d6370 0x56297d2e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d2e6740 0x56297d2e8b60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2db1c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 --2- 172.21.15.28:0/4091846422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d09a0 0x56297d2dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f0038ff9640 0 client.16614 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 >> 172.21.15.28:0/4091846422 conn(0x56297d2cc010 msgr2=0x56297d2cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.042+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 shutdown_connections 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.043+0000 7f005a288740 1 -- 172.21.15.28:0/4091846422 wait complete. 2024-09-19T09:37:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.043+0000 7f005a288740 20 client.16614 trim_cache size 0 max 16384 2024-09-19T09:37:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.057+0000 7f005a288740 1 Processor -- start 2024-09-19T09:37:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 -- start start 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00501ab180 con 0x56297d2d12e0 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00501acd50 con 0x56297d2d09a0 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f005a288740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00501ab850 con 0x56297d2d6370 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47052/0 (socket says 172.21.15.28:47052) 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 -- 172.21.15.28:0/1672173882 learned_addr learned my addr 172.21.15.28:0/1672173882 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0055a8a640 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0055289640 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 -- 172.21.15.28:0/1672173882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 msgr2=0x56297d2cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 -- 172.21.15.28:0/1672173882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 msgr2=0x56297d2ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 -- 172.21.15.28:0/1672173882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297d2d1b30 con 0x56297d2d12e0 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.058+0000 7f0054a88640 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0040004840 tx=0x7f0040011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4679f3bc46faf73b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f003affd640 1 -- 172.21.15.28:0/1672173882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004001fd00 con 0x56297d2d12e0 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f003affd640 1 -- 172.21.15.28:0/1672173882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040004f90 con 0x56297d2d12e0 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f003affd640 1 -- 172.21.15.28:0/1672173882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040004290 con 0x56297d2d12e0 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 -- 172.21.15.28:0/1672173882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 msgr2=0x56297d2e21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0040004840 tx=0x7f0040011b60 comp rx=0 tx=0).stop 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 -- 172.21.15.28:0/1672173882 shutdown_connections 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2e21c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 --2- 172.21.15.28:0/1672173882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 -- 172.21.15.28:0/1672173882 >> 172.21.15.28:0/1672173882 conn(0x56297d2cc010 msgr2=0x56297d2d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 -- 172.21.15.28:0/1672173882 shutdown_connections 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.059+0000 7f005a288740 1 -- 172.21.15.28:0/1672173882 wait complete. 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 -- start start 2024-09-19T09:37:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 Processor -- start 2024-09-19T09:37:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00501ac680 con 0x56297d2d12e0 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00501abf20 con 0x56297d2d09a0 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00501a9160 con 0x56297d2d6370 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f005a288740 10 client.0 _ll_register_callbacks cb 0x7f004c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47056/0 (socket says 172.21.15.28:47056) 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 -- 172.21.15.28:0/1305484315 learned_addr learned my addr 172.21.15.28:0/1305484315 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0055a8a640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0055289640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 -- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 msgr2=0x56297d2e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2e0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 -- 172.21.15.28:0/1305484315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 msgr2=0x56297d2df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2df320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.060+0000 7f0054a88640 1 -- 172.21.15.28:0/1305484315 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297d2e0e80 con 0x56297d2d12e0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0054a88640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f00400096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435f8ff4604ac43c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040004cb0 con 0x56297d2d12e0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f0040004050 con 0x56297d2d12e0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0040004f90 con 0x56297d2d12e0 2024-09-19T09:37:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0040005600 con 0x56297d2d12e0 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 10 client.16620 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56297d2e2190 con 0x56297d2d12e0 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040004f90 con 0x56297d2d12e0 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0040026070 con 0x56297d2d12e0 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 10 client.16620 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.061+0000 7f005a288740 20 client.16620 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d35a0a0 0x56297d35c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f005a288740 4 client.16620 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56297d23bd70 con 0x56297d35a0a0 2024-09-19T09:37:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f0055289640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d35a0a0 0x56297d35c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f0055289640 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d35a0a0 0x56297d35c4c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c40eac2a2ee9b4 server_cookie=9d351a581dd753e2 in_seq=0 out_seq=0 2024-09-19T09:37:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:34.062+0000 7f0038ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:34.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8d57fa640 1 -- 172.21.15.28:0/1067254175 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556ac9358d20 con 0x556ac9472490 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8d57fa640 10 client.16602 handle_command_reply: tid=0 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 client.16602 shutdown 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 msgr2=0x556ac93ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x556ac93f4e60 tx=0x7fb8dc0a4fc0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 shutdown_connections 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac93f6940 0x556ac93fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x556ac9472490 0x556ac94748b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ac93ec080 0x556ac93ff1c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 --2- 172.21.15.28:0/1067254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ac93eb760 0x556ac93fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8d57fa640 0 client.16602 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 >> 172.21.15.28:0/1067254175 conn(0x556ac93e7cf0 msgr2=0x556ac93f3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 shutdown_connections 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 1 -- 172.21.15.28:0/1067254175 wait complete. 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.393+0000 7fb8f60a7740 20 client.16602 trim_cache size 0 max 16384 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 35, 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 22, 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:37:34.397 INFO:teuthology.orchestra.run.smithi028.stdout: "a2c74700": { 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 22, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 149.8, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:34.398 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:34.399 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 msgr2=0x7fb8d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 0x7fb8d80a5780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40318b0 tx=0x7fb8e404a000 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 msgr2=0x7fb8ec1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ee80 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 shutdown_connections 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8d80a32c0 0x7fb8d80a5780 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8ec1082e0 0x7fb8ec19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8ec103760 0x7fb8ec1a0920 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 --2- 172.21.15.28:0/2507564073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8ec102db0 0x7fb8ec1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 >> 172.21.15.28:0/2507564073 conn(0x7fb8ec0fc7d0 msgr2=0x7fb8ec100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:34.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 shutdown_connections 2024-09-19T09:37:34.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:34.394+0000 7fb8f3cce640 1 -- 172.21.15.28:0/2507564073 wait complete. 2024-09-19T09:37:35.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a2c74700', quiesce took 14.1 seconds. Will release after: 15.700000000000001 2024-09-19T09:37:35.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:35 smithi028 ceph-mon[30972]: pgmap v486: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 972 KiB/s wr, 325 op/s 2024-09-19T09:37:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:35 smithi078 ceph-mon[35619]: pgmap v486: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 972 KiB/s wr, 325 op/s 2024-09-19T09:37:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:35 smithi038 ceph-mon[35271]: pgmap v486: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 972 KiB/s wr, 325 op/s 2024-09-19T09:37:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.067+0000 7f0038ff9640 1 -- 172.21.15.28:0/1305484315 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56297d23bd70 con 0x56297d35a0a0 2024-09-19T09:37:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.067+0000 7f0038ff9640 10 client.16620 handle_command_reply: tid=0 2024-09-19T09:37:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.067+0000 7f005a288740 1 client.16620 shutdown 2024-09-19T09:37:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.067+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 msgr2=0x56297d2df860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.067+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f00400096c0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 shutdown_connections 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56297d35a0a0 0x56297d35c4c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56297d2d6370 0x56297d2e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f0038ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56297d2d12e0 0x56297d2df860 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 --2- 172.21.15.28:0/1305484315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56297d2d09a0 0x56297d2df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 >> 172.21.15.28:0/1305484315 conn(0x56297d2cc010 msgr2=0x56297d2ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 shutdown_connections 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 1 -- 172.21.15.28:0/1305484315 wait complete. 2024-09-19T09:37:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f005a288740 20 client.16620 trim_cache size 0 max 16384 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 msgr2=0x7f003c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 0x7f003c0a5780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56297d2d1d10 tx=0x7f004c005f90 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 msgr2=0x7f00501a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.068+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0048005e30 tx=0x7f0048004ab0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 shutdown_connections 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f003c0a32c0 0x7f003c0a5780 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00501016c0 0x7f00501a1600 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050100d20 0x7f005019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 --2- 172.21.15.28:0/2471328807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00501003e0 0x7f005019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 >> 172.21.15.28:0/2471328807 conn(0x7f00500fc1f0 msgr2=0x7f00500fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 shutdown_connections 2024-09-19T09:37:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:36.069+0000 7f00574ed640 1 -- 172.21.15.28:0/2471328807 wait complete. 2024-09-19T09:37:37.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:37 smithi078 ceph-mon[35619]: pgmap v487: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 773 KiB/s wr, 229 op/s 2024-09-19T09:37:37.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:37 smithi038 ceph-mon[35271]: pgmap v487: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 773 KiB/s wr, 229 op/s 2024-09-19T09:37:37.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:37 smithi028 ceph-mon[30972]: pgmap v487: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 773 KiB/s wr, 229 op/s 2024-09-19T09:37:39.058 INFO:tasks.workunit.client.0.smithi028.stdout: 115 4808 0 3072 0 2012 0 2024-09-19T09:37:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:39 smithi078 ceph-mon[35619]: pgmap v488: 97 pgs: 97 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 876 KiB/s wr, 277 op/s 2024-09-19T09:37:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:39 smithi038 ceph-mon[35271]: pgmap v488: 97 pgs: 97 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 876 KiB/s wr, 277 op/s 2024-09-19T09:37:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:39 smithi028 ceph-mon[30972]: pgmap v488: 97 pgs: 97 active+clean; 378 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 876 KiB/s wr, 277 op/s 2024-09-19T09:37:41.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:41 smithi078 ceph-mon[35619]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 424 op/s 2024-09-19T09:37:41.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:41 smithi038 ceph-mon[35271]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 424 op/s 2024-09-19T09:37:41.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:41 smithi028 ceph-mon[30972]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 424 op/s 2024-09-19T09:37:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:43 smithi078 ceph-mon[35619]: pgmap v490: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 461 op/s 2024-09-19T09:37:43.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:43 smithi038 ceph-mon[35271]: pgmap v490: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 461 op/s 2024-09-19T09:37:43.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:43 smithi028 ceph-mon[30972]: pgmap v490: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 461 op/s 2024-09-19T09:37:45.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:45 smithi078 ceph-mon[35619]: pgmap v491: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 450 op/s 2024-09-19T09:37:45.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:45 smithi038 ceph-mon[35271]: pgmap v491: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 450 op/s 2024-09-19T09:37:45.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:45 smithi028 ceph-mon[30972]: pgmap v491: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 450 op/s 2024-09-19T09:37:47.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:47 smithi078 ceph-mon[35619]: pgmap v492: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 977 KiB/s wr, 509 op/s 2024-09-19T09:37:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:47 smithi038 ceph-mon[35271]: pgmap v492: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 977 KiB/s wr, 509 op/s 2024-09-19T09:37:47.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:47 smithi028 ceph-mon[30972]: pgmap v492: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 977 KiB/s wr, 509 op/s 2024-09-19T09:37:49.058 INFO:tasks.workunit.client.0.smithi028.stdout: 115 106861 0 78818 0 56309 0 2024-09-19T09:37:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 -- 172.21.15.28:0/729118245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c107820 msgr2=0x7fb82c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 --2- 172.21.15.28:0/729118245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c107820 0x7fb82c109c90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb824009ca0 tx=0x7fb82402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 -- 172.21.15.28:0/729118245 shutdown_connections 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 --2- 172.21.15.28:0/729118245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c104ef0 0x7fb82c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 --2- 172.21.15.28:0/729118245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c107820 0x7fb82c109c90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 --2- 172.21.15.28:0/729118245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 0x7fb82c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 -- 172.21.15.28:0/729118245 >> 172.21.15.28:0/729118245 conn(0x7fb82c0fc1c0 msgr2=0x7fb82c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 -- 172.21.15.28:0/729118245 shutdown_connections 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.171+0000 7fb833453640 1 -- 172.21.15.28:0/729118245 wait complete. 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 Processor -- start 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 -- start start 2024-09-19T09:37:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 0x7fb82c101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c107820 0x7fb82c100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c10d800 con 0x7fb82c104ef0 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c10d680 con 0x7fb82c107820 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb833453640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c10d980 con 0x7fb82c1025c0 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb8309ee640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb8309ee640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47512/0 (socket says 172.21.15.28:47512) 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb8309ee640 1 -- 172.21.15.28:0/209212899 learned_addr learned my addr 172.21.15.28:0/209212899 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.172+0000 7fb8311ef640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 0x7fb82c101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8319f0640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c107820 0x7fb82c100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 -- 172.21.15.28:0/209212899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 msgr2=0x7fb82c101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 0x7fb82c101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 -- 172.21.15.28:0/209212899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c107820 msgr2=0x7fb82c100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c107820 0x7fb82c100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 -- 172.21.15.28:0/209212899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c101340 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8309ee640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00d770 tx=0x7fb81c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8227fc640 1 -- 172.21.15.28:0/209212899 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c017070 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8227fc640 1 -- 172.21.15.28:0/209212899 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c009c30 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c1a1fa0 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb8227fc640 1 -- 172.21.15.28:0/209212899 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00f620 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.173+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a24b0 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8227fc640 1 -- 172.21.15.28:0/209212899 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb81c01a070 con 0x7fb82c104ef0 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 Processor -- start 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 -- start start 2024-09-19T09:37:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 0x562e65e07da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65e0a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562e65d662f0 con 0x562e65e029a0 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e65d0e270 con 0x562e65e08370 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8361ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e65d622b0 con 0x562e65e032e0 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8319f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8319f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47520/0 (socket says 172.21.15.28:47520) 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.176+0000 7fb8319f0640 1 -- 172.21.15.28:0/2423961692 learned_addr learned my addr 172.21.15.28:0/2423961692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8227fc640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 0x7fb8180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8309ee640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65e0a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8311ef640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 0x562e65e07da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 -- 172.21.15.28:0/2423961692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 msgr2=0x562e65e07da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 0x562e65e07da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 -- 172.21.15.28:0/2423961692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 msgr2=0x562e65e0a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65e0a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 -- 172.21.15.28:0/2423961692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e65d0d340 con 0x562e65e029a0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8311ef640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 0x562e65e07da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8227fc640 1 -- 172.21.15.28:0/209212899 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb81c0cf370 con 0x7fb82c104ef0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 0x7fb8180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8319f0640 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb824005cc0 tx=0x7fb824005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543f8ff4e10bdc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.177+0000 7fb8177fe640 1 -- 172.21.15.28:0/2423961692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824041040 con 0x562e65e029a0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8177fe640 1 -- 172.21.15.28:0/2423961692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824036070 con 0x562e65e029a0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8177fe640 1 -- 172.21.15.28:0/2423961692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82402f900 con 0x562e65e029a0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8319f0640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 0x7fb8180a5780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb82404c000 tx=0x7fb82404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 -- 172.21.15.28:0/2423961692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 msgr2=0x562e65e02da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb824005cc0 tx=0x7fb824005ef0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 -- 172.21.15.28:0/2423961692 shutdown_connections 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65e0a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e032e0 0x562e65e07da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 --2- 172.21.15.28:0/2423961692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e02da0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 -- 172.21.15.28:0/2423961692 >> 172.21.15.28:0/2423961692 conn(0x562e65dfe010 msgr2=0x562e65dfe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 -- 172.21.15.28:0/2423961692 shutdown_connections 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.178+0000 7fb8361ef740 1 -- 172.21.15.28:0/2423961692 wait complete. 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 -- start start 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 Processor -- start 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65e0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562e65e0b340 con 0x562e65e08370 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e65cef290 con 0x562e65e032e0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e65e0b4c0 con 0x562e65e029a0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8361ef740 10 client.0 _ll_register_callbacks cb 0x562e65d20140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8319f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8319f0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47534/0 (socket says 172.21.15.28:47534) 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8311ef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65e0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8319f0640 1 -- 172.21.15.28:0/3733284286 learned_addr learned my addr 172.21.15.28:0/3733284286 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.179+0000 7fb8309ee640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 -- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 msgr2=0x562e65e0cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65e0cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 -- 172.21.15.28:0/3733284286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 msgr2=0x562e65e0d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e0d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 -- 172.21.15.28:0/3733284286 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e65a79210 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8319f0640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bf0 tx=0x7fb824005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726fd397cd9baf91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824058020 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb824004000 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb824036070 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb824041040 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8361ef740 10 client.16632 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.180+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562e65b45620 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824036070 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb824030b10 con 0x562e65e08370 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 10 client.16632 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 20 client.16632 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e18470 0x562e65e1a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 4 client.16632 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e65d73a20 con 0x562e65e18470 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8311ef640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e18470 0x562e65e1a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb8311ef640 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e18470 0x562e65e1a860 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f31ece130a75657 server_cookie=7a9dd4f7359bfc10 in_seq=0 out_seq=0 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.181+0000 7fb814ff9640 10 client.16632 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb814ff9640 1 -- 172.21.15.28:0/3733284286 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e65d73a20 con 0x562e65e18470 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb814ff9640 10 client.16632 handle_command_reply: tid=0 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 client.16632 shutdown 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 msgr2=0x562e65e14170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb824009bf0 tx=0x7fb824005d80 comp rx=0 tx=0).stop 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 shutdown_connections 2024-09-19T09:37:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e0d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e18470 0x562e65e1a860 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e08370 0x562e65e14170 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 --2- 172.21.15.28:0/3733284286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65e0cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb814ff9640 0 client.16632 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 >> 172.21.15.28:0/3733284286 conn(0x562e65dfe010 msgr2=0x562e65dff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.182+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 shutdown_connections 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.183+0000 7fb8361ef740 1 -- 172.21.15.28:0/3733284286 wait complete. 2024-09-19T09:37:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.183+0000 7fb8361ef740 20 client.16632 trim_cache size 0 max 16384 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 Processor -- start 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 -- start start 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65dfeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65dff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ab2f0 con 0x562e65e032e0 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1acec0 con 0x562e65e08370 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8361ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ab9c0 con 0x562e65e029a0 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8309ee640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8309ee640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47548/0 (socket says 172.21.15.28:47548) 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.197+0000 7fb8309ee640 1 -- 172.21.15.28:0/1430316554 learned_addr learned my addr 172.21.15.28:0/1430316554 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8311ef640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65dfeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 -- 172.21.15.28:0/1430316554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 msgr2=0x562e65dfeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65dfeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8319f0640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65dff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 -- 172.21.15.28:0/1430316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 msgr2=0x562e65dff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65dff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 -- 172.21.15.28:0/1430316554 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e65e1abd0 con 0x562e65e032e0 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8311ef640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65dfeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8319f0640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65dff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb8309ee640 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb81c010fd0 tx=0x7fb81c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c364e7f41de04172 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb816ffd640 1 -- 172.21.15.28:0/1430316554 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c021070 con 0x562e65e032e0 2024-09-19T09:37:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb816ffd640 1 -- 172.21.15.28:0/1430316554 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c0a4630 con 0x562e65e032e0 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.198+0000 7fb816ffd640 1 -- 172.21.15.28:0/1430316554 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0a41a0 con 0x562e65e032e0 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- 172.21.15.28:0/1430316554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 msgr2=0x562e65e14130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb81c010fd0 tx=0x7fb81c086040 comp rx=0 tx=0).stop 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- 172.21.15.28:0/1430316554 shutdown_connections 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e08370 0x562e65dff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e032e0 0x562e65e14130 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- 172.21.15.28:0/1430316554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e029a0 0x562e65dfeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- 172.21.15.28:0/1430316554 >> 172.21.15.28:0/1430316554 conn(0x562e65dfe010 msgr2=0x562e65e0cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- 172.21.15.28:0/1430316554 shutdown_connections 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- 172.21.15.28:0/1430316554 wait complete. 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 -- start start 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 Processor -- start 2024-09-19T09:37:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.199+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 0x562e65e12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ac7f0 con 0x562e65e029a0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ac090 con 0x562e65e032e0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1a92d0 con 0x562e65e08370 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 10 client.0 _ll_register_callbacks cb 0x7fb81c09b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47564/0 (socket says 172.21.15.28:47564) 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 -- 172.21.15.28:0/723320575 learned_addr learned my addr 172.21.15.28:0/723320575 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8319f0640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 0x562e65e12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 -- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 msgr2=0x562e65e12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8309ee640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 0x562e65e12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 -- 172.21.15.28:0/723320575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 msgr2=0x562e65e11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e11900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 -- 172.21.15.28:0/723320575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e65e12f20 con 0x562e65e029a0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8319f0640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 0x562e65e12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8309ee640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8311ef640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb828005eb0 tx=0x7fb828004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea68acecd2c2c471 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828027630 con 0x562e65e029a0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb8280277d0 con 0x562e65e029a0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c0a4630 con 0x562e65e029a0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb82802e070 con 0x562e65e029a0 2024-09-19T09:37:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 10 client.16638 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.200+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562e65e14270 con 0x562e65e029a0 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c0a4630 con 0x562e65e029a0 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb828010b60 con 0x562e65e029a0 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 10 client.16638 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 20 client.16638 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e835d0 0x562e65e859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 4 client.16638 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e65d6dd70 con 0x562e65e835d0 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.201+0000 7fb8309ee640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e835d0 0x562e65e859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.202+0000 7fb814ff9640 10 client.16638 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.202+0000 7fb8309ee640 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e835d0 0x562e65e859f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b306fe25d415add0 server_cookie=d63dd2da10c7ed43 in_seq=0 out_seq=0 2024-09-19T09:37:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb814ff9640 1 -- 172.21.15.28:0/723320575 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e65d6dd70 con 0x562e65e835d0 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb814ff9640 10 client.16638 handle_command_reply: tid=0 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 client.16638 shutdown 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 msgr2=0x562e65e113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb828005eb0 tx=0x7fb828004560 comp rx=0 tx=0).stop 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 shutdown_connections 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e65e032e0 0x562e65e11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x562e65e835d0 0x562e65e859f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e65e08370 0x562e65e12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 --2- 172.21.15.28:0/723320575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562e65e029a0 0x562e65e113c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 >> 172.21.15.28:0/723320575 conn(0x562e65dfe010 msgr2=0x562e65dfc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb814ff9640 0 client.16638 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 shutdown_connections 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 1 -- 172.21.15.28:0/723320575 wait complete. 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb8361ef740 20 client.16638 trim_cache size 0 max 16384 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:37:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:37:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 msgr2=0x7fb8180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 0x7fb8180a5780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb82404c000 tx=0x7fb82404d000 comp rx=0 tx=0).stop 2024-09-19T09:37:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 msgr2=0x7fb82c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.226+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00d770 tx=0x7fb81c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 shutdown_connections 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb8180a32c0 0x7fb8180a5780 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb82c107820 0x7fb82c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb82c104ef0 0x7fb82c100570 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 --2- 172.21.15.28:0/209212899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb82c1025c0 0x7fb82c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 >> 172.21.15.28:0/209212899 conn(0x7fb82c0fc1c0 msgr2=0x7fb82c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 shutdown_connections 2024-09-19T09:37:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:37:49.227+0000 7fb833453640 1 -- 172.21.15.28:0/209212899 wait complete. 2024-09-19T09:37:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:49 smithi078 ceph-mon[35619]: pgmap v493: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 919 KiB/s wr, 571 op/s 2024-09-19T09:37:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:49 smithi038 ceph-mon[35271]: pgmap v493: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 919 KiB/s wr, 571 op/s 2024-09-19T09:37:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:49 smithi028 ceph-mon[30972]: pgmap v493: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 919 KiB/s wr, 571 op/s 2024-09-19T09:37:50.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:37:51.199 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:51.567 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:51 smithi028 ceph-mon[30972]: pgmap v494: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 886 KiB/s wr, 530 op/s 2024-09-19T09:37:51.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:51 smithi078 ceph-mon[35619]: pgmap v494: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 886 KiB/s wr, 530 op/s 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 -- 172.21.15.28:0/3881259179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084101de0 msgr2=0x7f80841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 --2- 172.21.15.28:0/3881259179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084101de0 0x7f80841021c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8074009900 tx=0x7f807402a3f0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 -- 172.21.15.28:0/3881259179 shutdown_connections 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 --2- 172.21.15.28:0/3881259179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084102790 0x7f80841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 --2- 172.21.15.28:0/3881259179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084106f10 0x7f808410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 --2- 172.21.15.28:0/3881259179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084101de0 0x7f80841021c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 -- 172.21.15.28:0/3881259179 >> 172.21.15.28:0/3881259179 conn(0x7f80840fd8c0 msgr2=0x7f80840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 -- 172.21.15.28:0/3881259179 shutdown_connections 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.648+0000 7f8089025640 1 -- 172.21.15.28:0/3881259179 wait complete. 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 Processor -- start 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 -- start start 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 0x7f808419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 0x7f808419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8084111fe0 con 0x7f8084102790 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8084111e60 con 0x7f8084106f10 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.649+0000 7f8089025640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112160 con 0x7f8084101de0 2024-09-19T09:37:51.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47572/0 (socket says 172.21.15.28:47572) 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 -- 172.21.15.28:0/866426504 learned_addr learned my addr 172.21.15.28:0/866426504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808359e640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 0x7f808419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f8082d9d640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 0x7f808419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 -- 172.21.15.28:0/866426504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 msgr2=0x7f808419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 0x7f808419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 -- 172.21.15.28:0/866426504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 msgr2=0x7f808419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 0x7f808419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 -- 172.21.15.28:0/866426504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808419f610 con 0x7f8084102790 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808359e640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 0x7f808419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f8082d9d640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 0x7f808419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f808259c640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f807800c920 tx=0x7f807800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:51.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078007c70 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078007e10 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808419f900 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078018890 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.650+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a71e0 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.651+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80841021c0 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.653+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8078018a70 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.653+0000 7f806bfff640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 0x7f806c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.653+0000 7f8082d9d640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 0x7f806c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.653+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f80780cfae0 con 0x7f8084102790 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.654+0000 7f8082d9d640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 0x7f806c0a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8074000c00 tx=0x7f8074005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:51.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.658+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8078014030 con 0x7f8084102790 2024-09-19T09:37:51.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:51 smithi038 ceph-mon[35271]: pgmap v494: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 886 KiB/s wr, 530 op/s 2024-09-19T09:37:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.871+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80841a7440 con 0x7f8084102790 2024-09-19T09:37:51.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.872+0000 7f806bfff640 1 -- 172.21.15.28:0/866426504 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11958 (secure 0 0 0) 0x7f807800de20 con 0x7f8084102790 2024-09-19T09:37:51.874 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:37:51.875 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":35,"btime":"2024-09-19T09:37:05:233386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:37:05.233357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:37:51.875 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 35 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 msgr2=0x7f806c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 0x7f806c0a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8074000c00 tx=0x7f8074005a20 comp rx=0 tx=0).stop 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 msgr2=0x7f808419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f807800c920 tx=0x7f807800cde0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 shutdown_connections 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f806c0a3240 0x7f806c0a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8084106f10 0x7f808419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8084102790 0x7f808419aba0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 --2- 172.21.15.28:0/866426504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8084101de0 0x7f808419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:51.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 >> 172.21.15.28:0/866426504 conn(0x7f80840fd8c0 msgr2=0x7f80840fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:51.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 shutdown_connections 2024-09-19T09:37:51.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:51.875+0000 7f8089025640 1 -- 172.21.15.28:0/866426504 wait complete. 2024-09-19T09:37:52.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:52 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/866426504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:52.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:52 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/866426504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:52.705 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:52 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/866426504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:37:52.737 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id a2c74700 --release --await' 2024-09-19T09:37:52.737 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id a2c74700 --release --await 2024-09-19T09:37:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:37:53.407 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:53 smithi028 ceph-mon[30972]: pgmap v495: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 384 KiB/s wr, 375 op/s 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 -- 172.21.15.28:0/4165886568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8103740 msgr2=0x7fd1e8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 --2- 172.21.15.28:0/4165886568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8103740 0x7fd1e8107b80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009930 tx=0x7fd1d002a420 comp rx=0 tx=0).stop 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 -- 172.21.15.28:0/4165886568 shutdown_connections 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 --2- 172.21.15.28:0/4165886568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8108290 0x7fd1e810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 --2- 172.21.15.28:0/4165886568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8103740 0x7fd1e8107b80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 --2- 172.21.15.28:0/4165886568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8102d90 0x7fd1e8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 -- 172.21.15.28:0/4165886568 >> 172.21.15.28:0/4165886568 conn(0x7fd1e80fc7d0 msgr2=0x7fd1e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 -- 172.21.15.28:0/4165886568 shutdown_connections 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.405+0000 7fd1ee275640 1 -- 172.21.15.28:0/4165886568 wait complete. 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 Processor -- start 2024-09-19T09:37:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 -- start start 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 0x7fd1e8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 0x7fd1e819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81120b0 con 0x7fd1e8102d90 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e8111f30 con 0x7fd1e8103740 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ee275640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e8112230 con 0x7fd1e8108290 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43706/0 (socket says 172.21.15.28:43706) 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1e7fff640 1 -- 172.21.15.28:0/2378384021 learned_addr learned my addr 172.21.15.28:0/2378384021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1dffff640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 0x7fd1e8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.406+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 0x7fd1e819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 -- 172.21.15.28:0/2378384021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 msgr2=0x7fd1e819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 0x7fd1e819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 -- 172.21.15.28:0/2378384021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 msgr2=0x7fd1e8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 0x7fd1e8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 -- 172.21.15.28:0/2378384021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e819d570 con 0x7fd1e8102d90 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 0x7fd1e819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1dffff640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 0x7fd1e8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800b4a0 tx=0x7fd1d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e5ffb640 1 -- 172.21.15.28:0/2378384021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8014070 con 0x7fd1e8102d90 2024-09-19T09:37:53.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e5ffb640 1 -- 172.21.15.28:0/2378384021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d8010af0 con 0x7fd1e8102d90 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1e5ffb640 1 -- 172.21.15.28:0/2378384021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8017d10 con 0x7fd1e8102d90 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e819d860 con 0x7fd1e8102d90 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.407+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e81a5140 con 0x7fd1e8102d90 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1e5ffb640 1 -- 172.21.15.28:0/2378384021 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd1d8017070 con 0x7fd1e8102d90 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 Processor -- start 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 -- start start 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc13850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bdecb74620 con 0x55bdecc1e650 2024-09-19T09:37:53.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bdecb1be50 con 0x55bdecc13d90 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1f064e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdecb705e0 con 0x55bdecc13470 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1dffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1dffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43708/0 (socket says 172.21.15.28:43708) 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1dffff640 1 -- 172.21.15.28:0/17809190 learned_addr learned my addr 172.21.15.28:0/17809190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.410+0000 7fd1ec812640 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc13850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1e5ffb640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 0x7fd1d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 -- 172.21.15.28:0/17809190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 msgr2=0x55bdecc13850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc13850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 -- 172.21.15.28:0/17809190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 msgr2=0x55bdecc1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 0x7fd1d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 -- 172.21.15.28:0/17809190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdecb0cd40 con 0x55bdecc1e650 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1e5ffb640 1 -- 172.21.15.28:0/2378384021 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fd1d80ced60 con 0x7fd1e8102d90 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1dffff640 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c00 tx=0x7fd1d0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fece3ff3330ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 0x7fd1d40a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00124d0 tx=0x7fd1e0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1de7fc640 1 -- 172.21.15.28:0/17809190 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d003f070 con 0x55bdecc1e650 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1de7fc640 1 -- 172.21.15.28:0/17809190 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d00035f0 con 0x55bdecc1e650 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.411+0000 7fd1de7fc640 1 -- 172.21.15.28:0/17809190 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0035070 con 0x55bdecc1e650 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- 172.21.15.28:0/17809190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 msgr2=0x55bdecc20a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c00 tx=0x7fd1d0031700 comp rx=0 tx=0).stop 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- 172.21.15.28:0/17809190 shutdown_connections 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc20a40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 --2- 172.21.15.28:0/17809190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc13850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- 172.21.15.28:0/17809190 >> 172.21.15.28:0/17809190 conn(0x55bdecc0fa00 msgr2=0x55bdecc0fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- 172.21.15.28:0/17809190 shutdown_connections 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- 172.21.15.28:0/17809190 wait complete. 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.412+0000 7fd1f064e740 1 -- start start 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 Processor -- start 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13470 0x55bdecc1b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13d90 0x55bdecc1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bdecc21220 con 0x55bdecc1e650 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bdecafcf10 con 0x55bdecc13470 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdecc213a0 con 0x55bdecc13d90 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1f064e740 10 client.0 _ll_register_callbacks cb 0x55bdecb20db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:53.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43716/0 (socket says 172.21.15.28:43716) 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 -- 172.21.15.28:0/2495254228 learned_addr learned my addr 172.21.15.28:0/2495254228 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1dffff640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13d90 0x55bdecc1ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13470 0x55bdecc1b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 -- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13d90 msgr2=0x55bdecc1ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13d90 0x55bdecc1ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 -- 172.21.15.28:0/2495254228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13470 msgr2=0x55bdecc1b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13470 0x55bdecc1b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.413+0000 7fd1ec812640 1 -- 172.21.15.28:0/2495254228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdec883580 con 0x55bdecc1e650 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1ec812640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0010ef0 tx=0x7fd1e0007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d667d6011c89cbb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:53.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0026470 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fd1e0026610 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d00035f0 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd1e0026810 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1f064e740 10 client.16656 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bdec94d960 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.414+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d00035f0 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd1e00269b0 con 0x55bdecc1e650 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 10 client.16656 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 10 client.16656 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 20 client.16656 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc2e1f0 0x55bdecc30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 4 client.16656 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdecb81b90 con 0x55bdecc2e1f0 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.415+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc2e1f0 0x55bdecc30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1c77fe640 10 client.16656 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc2e1f0 0x55bdecc30690 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751416249f24d463 server_cookie=23605ee00a68b456 in_seq=0 out_seq=0 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1c77fe640 1 -- 172.21.15.28:0/2495254228 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdecb81b90 con 0x55bdecc2e1f0 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1c77fe640 10 client.16656 handle_command_reply: tid=0 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1f064e740 1 client.16656 shutdown 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 msgr2=0x55bdecc1bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.416+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0010ef0 tx=0x7fd1e0007590 comp rx=0 tx=0).stop 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 shutdown_connections 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc2e1f0 0x55bdecc30690 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc1bf80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13d90 0x55bdecc1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1c77fe640 0 client.16656 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 --2- 172.21.15.28:0/2495254228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13470 0x55bdecc1b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 >> 172.21.15.28:0/2495254228 conn(0x55bdecc0fa00 msgr2=0x55bdecc0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 shutdown_connections 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 1 -- 172.21.15.28:0/2495254228 wait complete. 2024-09-19T09:37:53.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.417+0000 7fd1f064e740 20 client.16656 trim_cache size 0 max 16384 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 Processor -- start 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 -- start start 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc0a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc35f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81ab220 con 0x55bdecc1e650 2024-09-19T09:37:53.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81acdf0 con 0x55bdecc13d90 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1f064e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81ab8f0 con 0x55bdecc13470 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43732/0 (socket says 172.21.15.28:43732) 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.434+0000 7fd1ec812640 1 -- 172.21.15.28:0/1325473399 learned_addr learned my addr 172.21.15.28:0/1325473399 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1dffff640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc35f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc0a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 -- 172.21.15.28:0/1325473399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 msgr2=0x55bdecc0a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc0a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 -- 172.21.15.28:0/1325473399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 msgr2=0x55bdecc35f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc35f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 -- 172.21.15.28:0/1325473399 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdecc0f830 con 0x55bdecc1e650 2024-09-19T09:37:53.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ec812640 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0000f80 tx=0x7fd1e0007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887682d317362183 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ddffb640 1 -- 172.21.15.28:0/1325473399 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0007780 con 0x55bdecc1e650 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ddffb640 1 -- 172.21.15.28:0/1325473399 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e0021070 con 0x55bdecc1e650 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.435+0000 7fd1ddffb640 1 -- 172.21.15.28:0/1325473399 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0026640 con 0x55bdecc1e650 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- 172.21.15.28:0/1325473399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 msgr2=0x55bdecc364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0000f80 tx=0x7fd1e0007970 comp rx=0 tx=0).stop 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- 172.21.15.28:0/1325473399 shutdown_connections 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc35f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc364a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 --2- 172.21.15.28:0/1325473399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- 172.21.15.28:0/1325473399 >> 172.21.15.28:0/1325473399 conn(0x55bdecc0fa00 msgr2=0x55bdecc09f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- 172.21.15.28:0/1325473399 shutdown_connections 2024-09-19T09:37:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- 172.21.15.28:0/1325473399 wait complete. 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.436+0000 7fd1f064e740 1 -- start start 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 Processor -- start 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc26c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81ac720 con 0x55bdecc1e650 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81abfc0 con 0x55bdecc13d90 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81a9200 con 0x55bdecc13470 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1f064e740 10 client.0 _ll_register_callbacks cb 0x7fd1e0010b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43744/0 (socket says 172.21.15.28:43744) 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 -- 172.21.15.28:0/4105758819 learned_addr learned my addr 172.21.15.28:0/4105758819 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:37:53.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1dffff640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc26c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 -- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 msgr2=0x55bdecc26c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc26c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 -- 172.21.15.28:0/4105758819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 msgr2=0x55bdecc235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.437+0000 7fd1ec812640 1 -- 172.21.15.28:0/4105758819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdecc24240 con 0x55bdecc1e650 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1ec812640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000f910 tx=0x7fd1e00077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce91bfd44b64617 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0047060 con 0x55bdecc1e650 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fd1e001d950 con 0x55bdecc1e650 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e0021070 con 0x55bdecc1e650 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd1e001db50 con 0x55bdecc1e650 2024-09-19T09:37:53.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1f064e740 10 client.16662 fetch_fsmap learned FSMap version 35 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bdecc99eb0 con 0x55bdecc1e650 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e0021070 con 0x55bdecc1e650 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd1e0038070 con 0x55bdecc1e650 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1f064e740 10 client.16662 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.438+0000 7fd1f064e740 10 client.16662 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1f064e740 20 client.16662 populate_metadata read hostname 'smithi028' 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc9a4b0 0x55bdecc9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1f064e740 4 client.16662 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a2c74700", "release": true, "await": true}] 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a2c74700", "release": true, "await": true}) -- 0x55bdecb7c0a0 con 0x55bdecc9a4b0 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc9a4b0 0x55bdecc9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1e7fff640 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc9a4b0 0x55bdecc9c8d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51b2e574295f959 server_cookie=a01a36ebb1b155b in_seq=0 out_seq=0 2024-09-19T09:37:53.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:53.439+0000 7fd1c77fe640 10 client.16662 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:53 smithi078 ceph-mon[35619]: pgmap v495: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 384 KiB/s wr, 375 op/s 2024-09-19T09:37:53.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:53 smithi038 ceph-mon[35271]: pgmap v495: 97 pgs: 97 active+clean; 383 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 384 KiB/s wr, 375 op/s 2024-09-19T09:37:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1c77fe640 1 -- 172.21.15.28:0/4105758819 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bdecb7c0a0 con 0x55bdecc9a4b0 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1c77fe640 10 client.16662 handle_command_reply: tid=0 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 client.16662 shutdown 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 msgr2=0x55bdecc23b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000f910 tx=0x7fd1e00077f0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 shutdown_connections 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdecc13d90 0x55bdecc235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55bdecc9a4b0 0x55bdecc9c8d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1c77fe640 0 client.16662 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdecc1e650 0x55bdecc23b00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 --2- 172.21.15.28:0/4105758819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdecc13470 0x55bdecc26c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 >> 172.21.15.28:0/4105758819 conn(0x55bdecc0fa00 msgr2=0x55bdecc0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 shutdown_connections 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.352+0000 7fd1f064e740 1 -- 172.21.15.28:0/4105758819 wait complete. 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1f064e740 20 client.16662 trim_cache size 0 max 16384 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 35, 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 24, 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:37:54.356 INFO:teuthology.orchestra.run.smithi028.stdout: "a2c74700": { 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 24, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 149.8, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:37:54.357 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:37:54.358 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 msgr2=0x7fd1d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 0x7fd1d40a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00124d0 tx=0x7fd1e0004ab0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 msgr2=0x7fd1e81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800b4a0 tx=0x7fd1d800b960 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 shutdown_connections 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fd1d40a3240 0x7fd1d40a5700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1e8103740 0x7fd1e8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8108290 0x7fd1e819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 --2- 172.21.15.28:0/2378384021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1e8102d90 0x7fd1e81985c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.353+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 >> 172.21.15.28:0/2378384021 conn(0x7fd1e80fc7d0 msgr2=0x7fd1e8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:37:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.354+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 shutdown_connections 2024-09-19T09:37:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:37:54.354+0000 7fd1ee275640 1 -- 172.21.15.28:0/2378384021 wait complete. 2024-09-19T09:37:55.151 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a2c74700', total seconds elapsed: 34.0 2024-09-19T09:37:55.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:55 smithi028 ceph-mon[30972]: pgmap v496: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 348 KiB/s wr, 322 op/s 2024-09-19T09:37:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:55 smithi078 ceph-mon[35619]: pgmap v496: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 348 KiB/s wr, 322 op/s 2024-09-19T09:37:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:55 smithi038 ceph-mon[35271]: pgmap v496: 97 pgs: 97 active+clean; 379 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 348 KiB/s wr, 322 op/s 2024-09-19T09:37:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:57 smithi078 ceph-mon[35619]: pgmap v497: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 338 KiB/s wr, 246 op/s 2024-09-19T09:37:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:57 smithi038 ceph-mon[35271]: pgmap v497: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 338 KiB/s wr, 246 op/s 2024-09-19T09:37:57.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:57 smithi028 ceph-mon[30972]: pgmap v497: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 338 KiB/s wr, 246 op/s 2024-09-19T09:37:59.058 INFO:tasks.workunit.client.0.smithi028.stdout: 116 251721 166 188149 98 135041 406 2024-09-19T09:37:59.575 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:59 smithi028 ceph-mon[30972]: pgmap v498: 97 pgs: 97 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 347 KiB/s wr, 178 op/s 2024-09-19T09:37:59.575 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:37:59.576 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:37:59.576 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:37:59.576 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:37:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:37:59.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:59 smithi078 ceph-mon[35619]: pgmap v498: 97 pgs: 97 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 347 KiB/s wr, 178 op/s 2024-09-19T09:37:59.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:37:59.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:37:59.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:37:59.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:37:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:37:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:59 smithi038 ceph-mon[35271]: pgmap v498: 97 pgs: 97 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 347 KiB/s wr, 178 op/s 2024-09-19T09:37:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:37:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:37:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:37:59.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:37:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:38:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:01 smithi078 ceph-mon[35619]: pgmap v499: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 309 KiB/s wr, 127 op/s 2024-09-19T09:38:01.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:01 smithi038 ceph-mon[35271]: pgmap v499: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 309 KiB/s wr, 127 op/s 2024-09-19T09:38:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:01 smithi028 ceph-mon[30972]: pgmap v499: 97 pgs: 97 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 309 KiB/s wr, 127 op/s 2024-09-19T09:38:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1635943628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0109700 msgr2=0x7fb5b0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1635943628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0109700 0x7fb5b0109b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009ca0 tx=0x7fb5a802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1635943628 shutdown_connections 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1635943628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b0072d10 0x7fb5b0073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1635943628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b00736b0 0x7fb5b0119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1635943628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0109700 0x7fb5b0109b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.484+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1635943628 >> 172.21.15.28:0/1635943628 conn(0x7fb5b006ecd0 msgr2=0x7fb5b0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.485+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1635943628 shutdown_connections 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.485+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1635943628 wait complete. 2024-09-19T09:38:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 Processor -- start 2024-09-19T09:38:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 -- start start 2024-09-19T09:38:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b00736b0 0x7fb5b01a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 0x7fb5b01ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b011c5e0 con 0x7fb5b0072d10 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b011c460 con 0x7fb5b00736b0 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.486+0000 7fb5b55c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b011c760 con 0x7fb5b01ac090 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 0x7fb5b01ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58148/0 (socket says 172.21.15.28:58148) 2024-09-19T09:38:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5aeffd640 1 -- 172.21.15.28:0/1324226330 learned_addr learned my addr 172.21.15.28:0/1324226330 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1324226330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b00736b0 msgr2=0x7fb5b01a7030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:38:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.487+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b00736b0 0x7fb5b01a7030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.488+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 msgr2=0x7fb5b01a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.488+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.488+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1324226330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0110a50 con 0x7fb5b01ac090 2024-09-19T09:38:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.488+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.488+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 0x7fb5b01ae480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c970 tx=0x7fb5a400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.496+0000 7fb597fff640 1 -- 172.21.15.28:0/1324226330 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4007bb0 con 0x7fb5b01ac090 2024-09-19T09:38:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.496+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b0110d40 con 0x7fb5b01ac090 2024-09-19T09:38:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.496+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b0111290 con 0x7fb5b01ac090 2024-09-19T09:38:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.500+0000 7fb5b843f740 1 Processor -- start 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 -- start start 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573777099a0 0x557377709da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x557377711830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55737766d2f0 con 0x55737770f370 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557377615270 con 0x55737770a2e0 2024-09-19T09:38:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5b843f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5573776692b0 con 0x5573777099a0 2024-09-19T09:38:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb597fff640 1 -- 172.21.15.28:0/1324226330 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a4007d50 con 0x7fb5b01ac090 2024-09-19T09:38:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb597fff640 1 -- 172.21.15.28:0/1324226330 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4018770 con 0x7fb5b01ac090 2024-09-19T09:38:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5ae7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x557377711830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.501+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:48766/0 (socket says 172.21.15.28:48766) 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 -- 172.21.15.28:0/3453136806 learned_addr learned my addr 172.21.15.28:0/3453136806 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 -- 172.21.15.28:0/3453136806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573777099a0 msgr2=0x557377709da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573777099a0 0x557377709da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 -- 172.21.15.28:0/3453136806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 msgr2=0x557377711830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x557377711830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 -- 172.21.15.28:0/3453136806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557377614340 con 0x55737770a2e0 2024-09-19T09:38:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.502+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8031db0 tx=0x7fb5a80098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd9c33103d5d7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.504+0000 7fb597fff640 1 -- 172.21.15.28:0/1324226330 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb5a40213f0 con 0x7fb5b01ac090 2024-09-19T09:38:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.506+0000 7fb597fff640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 0x7fb5980a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.506+0000 7fb597fff640 1 -- 172.21.15.28:0/1324226330 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a40d0880 con 0x7fb5b01ac090 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.507+0000 7fb5957fa640 1 -- 172.21.15.28:0/3453136806 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8003e60 con 0x55737770a2e0 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.507+0000 7fb5957fa640 1 -- 172.21.15.28:0/3453136806 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a8036040 con 0x55737770a2e0 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.508+0000 7fb5b843f740 1 -- 172.21.15.28:0/3453136806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 msgr2=0x55737770eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.508+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8031db0 tx=0x7fb5a80098b0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.510+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x557377711830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:38:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.510+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 0x7fb5980a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.515+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 0x7fb5980a5890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0005d00 tx=0x7fb5a0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.518+0000 7fb5b843f740 1 -- 172.21.15.28:0/3453136806 shutdown_connections 2024-09-19T09:38:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.518+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770eda0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.518+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x557377711830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.518+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3453136806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5573777099a0 0x557377709da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.518+0000 7fb5b843f740 1 -- 172.21.15.28:0/3453136806 >> 172.21.15.28:0/3453136806 conn(0x557377705010 msgr2=0x557377705410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.535+0000 7fb5b843f740 1 -- 172.21.15.28:0/3453136806 shutdown_connections 2024-09-19T09:38:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.535+0000 7fb5b843f740 1 -- 172.21.15.28:0/3453136806 wait complete. 2024-09-19T09:38:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.535+0000 7fb5b843f740 1 -- start start 2024-09-19T09:38:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 Processor -- start 2024-09-19T09:38:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573777099a0 0x557377713af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x55737771b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557377712330 con 0x55737770f370 2024-09-19T09:38:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5573775f6290 con 0x5573777099a0 2024-09-19T09:38:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5573777124b0 con 0x55737770a2e0 2024-09-19T09:38:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5b843f740 10 client.0 _ll_register_callbacks cb 0x557377627140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.536+0000 7fb5ae7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:47114/0 (socket says 172.21.15.28:47114) 2024-09-19T09:38:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 -- 172.21.15.28:0/3466773015 learned_addr learned my addr 172.21.15.28:0/3466773015 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x55737771b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573777099a0 0x557377713af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 -- 172.21.15.28:0/3466773015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573777099a0 msgr2=0x557377713af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573777099a0 0x557377713af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 -- 172.21.15.28:0/3466773015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 msgr2=0x55737771b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x55737771b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 -- 172.21.15.28:0/3466773015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557377380210 con 0x55737770a2e0 2024-09-19T09:38:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.537+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00116a0 tx=0x7fb5a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f62387d758acc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.561+0000 7fb592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:38:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.561+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0022020 con 0x55737770a2e0 2024-09-19T09:38:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.575+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a0010790 con 0x55737770a2e0 2024-09-19T09:38:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.576+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a8036040 con 0x55737770a2e0 2024-09-19T09:38:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.576+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb5a0010990 con 0x55737770a2e0 2024-09-19T09:38:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.576+0000 7fb5b843f740 10 client.26209 fetch_fsmap learned FSMap version 35 2024-09-19T09:38:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.576+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55737744c620 con 0x55737770a2e0 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a8036040 con 0x55737770a2e0 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb5a002f020 con 0x55737770a2e0 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 10 client.26209 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 20 client.26209 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737771fc50 0x5573777220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 4 client.26209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:38:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55737767aa20 con 0x55737771fc50 2024-09-19T09:38:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.577+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737771fc50 0x5573777220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.578+0000 7fb592ff5640 10 client.26209 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.578+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737771fc50 0x5573777220f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b07d73bb367f6aa server_cookie=911ab3809b950507 in_seq=0 out_seq=0 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.594+0000 7fb592ff5640 1 -- 172.21.15.28:0/3466773015 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55737767aa20 con 0x55737771fc50 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb592ff5640 10 client.26209 handle_command_reply: tid=0 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 client.26209 shutdown 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 msgr2=0x557377714030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00116a0 tx=0x7fb5a00046c0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 shutdown_connections 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737771fc50 0x5573777220f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55737770f370 0x55737771b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377714030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 --2- 172.21.15.28:0/3466773015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573777099a0 0x557377713af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 >> 172.21.15.28:0/3466773015 conn(0x557377705010 msgr2=0x557377706820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb592ff5640 0 client.26209 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 shutdown_connections 2024-09-19T09:38:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.595+0000 7fb5b843f740 1 -- 172.21.15.28:0/3466773015 wait complete. 2024-09-19T09:38:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.596+0000 7fb5b843f740 20 client.26209 trim_cache size 0 max 16384 2024-09-19T09:38:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:03 smithi078 ceph-mon[35619]: pgmap v500: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 611 KiB/s wr, 73 op/s 2024-09-19T09:38:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:03 smithi038 ceph-mon[35271]: pgmap v500: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 611 KiB/s wr, 73 op/s 2024-09-19T09:38:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.703+0000 7fb5b843f740 1 Processor -- start 2024-09-19T09:38:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.707+0000 7fb5b843f740 1 -- start start 2024-09-19T09:38:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.707+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58178/0 (socket says 172.21.15.28:58178) 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5aeffd640 1 -- 172.21.15.28:0/1426923902 learned_addr learned my addr 172.21.15.28:0/1426923902 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 0x55737771b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00e380 con 0x5573777099a0 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00ff50 con 0x55737770a2e0 2024-09-19T09:38:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00ea50 con 0x55737770f370 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 0x55737771b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.708+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1426923902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 msgr2=0x55737770b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1426923902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 msgr2=0x55737770b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1426923902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557377713630 con 0x55737770f370 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.709+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 0x55737771b030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5a409ce60 tx=0x7fb5a400a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbbbb8e753e7061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.729+0000 7fb594ff9640 1 -- 172.21.15.28:0/1426923902 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40b5070 con 0x55737770f370 2024-09-19T09:38:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.731+0000 7fb594ff9640 1 -- 172.21.15.28:0/1426923902 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a40b9070 con 0x55737770f370 2024-09-19T09:38:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.731+0000 7fb594ff9640 1 -- 172.21.15.28:0/1426923902 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a409c320 con 0x55737770f370 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 msgr2=0x55737771b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 0x55737771b030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5a409ce60 tx=0x7fb5a400a490 comp rx=0 tx=0).stop 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 shutdown_connections 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770a2e0 0x55737770b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770f370 0x55737771b030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.732+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1426923902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x55737770b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.733+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 >> 172.21.15.28:0/1426923902 conn(0x557377705010 msgr2=0x557377713af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.735+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 shutdown_connections 2024-09-19T09:38:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.736+0000 7fb5b843f740 1 -- 172.21.15.28:0/1426923902 wait complete. 2024-09-19T09:38:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.736+0000 7fb5b843f740 1 -- start start 2024-09-19T09:38:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.737+0000 7fb5b843f740 1 Processor -- start 2024-09-19T09:38:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.737+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377718830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 0x557377719840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00f880 con 0x5573777099a0 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00f120 con 0x55737770f370 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00c360 con 0x55737770a2e0 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5aeffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58180/0 (socket says 172.21.15.28:58180) 2024-09-19T09:38:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5aeffd640 1 -- 172.21.15.28:0/1733430691 learned_addr learned my addr 172.21.15.28:0/1733430691 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5ae7fc640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377718830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5b843f740 10 client.0 _ll_register_callbacks cb 0x7fb5a00096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 0x557377719840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 msgr2=0x557377718830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377718830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.738+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1733430691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 msgr2=0x5573777182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.739+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.739+0000 7fb5af7fe640 1 -- 172.21.15.28:0/1733430691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557377719e50 con 0x55737770f370 2024-09-19T09:38:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.739+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.739+0000 7fb5af7fe640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 0x557377719840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400fe40 tx=0x7fb5a40a5b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544dff974857c55e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.761+0000 7fb592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:38:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.761+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40b5070 con 0x55737770f370 2024-09-19T09:38:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.761+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a40b9070 con 0x55737770f370 2024-09-19T09:38:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.761+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fb5a409c520 con 0x55737770f370 2024-09-19T09:38:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.761+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb5a40b10c0 con 0x55737770f370 2024-09-19T09:38:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 10 client.26228 fetch_fsmap learned FSMap version 35 2024-09-19T09:38:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55737771b160 con 0x55737770f370 2024-09-19T09:38:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a40b9070 con 0x55737770f370 2024-09-19T09:38:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb5a40bbc10 con 0x55737770f370 2024-09-19T09:38:03.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:03 smithi028 ceph-mon[30972]: pgmap v500: 97 pgs: 97 active+clean; 380 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 611 KiB/s wr, 73 op/s 2024-09-19T09:38:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 10 client.26228 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T09:38:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 20 client.26228 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737778ae10 0x55737778d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 4 client.26228 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:38:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.762+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557377674d70 con 0x55737778ae10 2024-09-19T09:38:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.763+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737778ae10 0x55737778d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.764+0000 7fb5aeffd640 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737778ae10 0x55737778d230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d103923ebee1ea19 server_cookie=313f19a5065d8e82 in_seq=0 out_seq=0 2024-09-19T09:38:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:03.765+0000 7fb592ff5640 10 client.26228 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:05 smithi078 ceph-mon[35619]: pgmap v501: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 716 KiB/s wr, 85 op/s 2024-09-19T09:38:05.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:05 smithi038 ceph-mon[35271]: pgmap v501: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 716 KiB/s wr, 85 op/s 2024-09-19T09:38:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:05 smithi028 ceph-mon[30972]: pgmap v501: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 716 KiB/s wr, 85 op/s 2024-09-19T09:38:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:07 smithi078 ceph-mon[35619]: pgmap v502: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 789 KiB/s wr, 104 op/s 2024-09-19T09:38:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:07 smithi038 ceph-mon[35271]: pgmap v502: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 789 KiB/s wr, 104 op/s 2024-09-19T09:38:07.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:07 smithi028 ceph-mon[30972]: pgmap v502: 97 pgs: 97 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 789 KiB/s wr, 104 op/s 2024-09-19T09:38:09.058 INFO:tasks.workunit.client.0.smithi028.stdout: 123 1590212 499 1151132 387 861004 1350 2024-09-19T09:38:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:09 smithi078 ceph-mon[35619]: pgmap v503: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-09-19T09:38:09.673 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:09 smithi028 ceph-mon[30972]: pgmap v503: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-09-19T09:38:09.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:09 smithi038 ceph-mon[35271]: pgmap v503: 97 pgs: 97 active+clean; 396 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-09-19T09:38:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.293+0000 7fb592ff5640 1 -- 172.21.15.28:0/1733430691 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557377674d70 con 0x55737778ae10 2024-09-19T09:38:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.293+0000 7fb592ff5640 10 client.26228 handle_command_reply: tid=0 2024-09-19T09:38:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.293+0000 7fb5b843f740 1 client.26228 shutdown 2024-09-19T09:38:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 msgr2=0x557377719840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 0x557377719840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400fe40 tx=0x7fb5a40a5b40 comp rx=0 tx=0).stop 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 shutdown_connections 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55737770f370 0x557377719840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55737778ae10 0x55737778d230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737770a2e0 0x557377718830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 --2- 172.21.15.28:0/1733430691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5573777099a0 0x5573777182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 >> 172.21.15.28:0/1733430691 conn(0x557377705010 msgr2=0x5573777039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb592ff5640 0 client.26228 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 shutdown_connections 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 1 -- 172.21.15.28:0/1733430691 wait complete. 2024-09-19T09:38:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.294+0000 7fb5b843f740 20 client.26228 trim_cache size 0 max 16384 2024-09-19T09:38:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:38:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:38:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:38:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:38:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 msgr2=0x7fb5980a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 0x7fb5980a5890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0005d00 tx=0x7fb5a0005c50 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 msgr2=0x7fb5b01ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 0x7fb5b01ae480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c970 tx=0x7fb5a400ce30 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 shutdown_connections 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fb5980a33d0 0x7fb5980a5890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b00736b0 0x7fb5b01a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b01ac090 0x7fb5b01ae480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 --2- 172.21.15.28:0/1324226330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5b0072d10 0x7fb5b01a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.321+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 >> 172.21.15.28:0/1324226330 conn(0x7fb5b006ecd0 msgr2=0x7fb5b0071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.322+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 shutdown_connections 2024-09-19T09:38:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:10.322+0000 7fb5b55c4640 1 -- 172.21.15.28:0/1324226330 wait complete. 2024-09-19T09:38:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:11 smithi078 ceph-mon[35619]: pgmap v504: 97 pgs: 97 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 660 op/s 2024-09-19T09:38:11.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:11 smithi038 ceph-mon[35271]: pgmap v504: 97 pgs: 97 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 660 op/s 2024-09-19T09:38:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:11 smithi028 ceph-mon[30972]: pgmap v504: 97 pgs: 97 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 660 op/s 2024-09-19T09:38:12.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:12 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:12 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:12 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:13.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:13 smithi038 ceph-mon[35271]: pgmap v505: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 731 op/s 2024-09-19T09:38:13.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:13 smithi028 ceph-mon[30972]: pgmap v505: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 731 op/s 2024-09-19T09:38:13.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:13 smithi078 ceph-mon[35619]: pgmap v505: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 731 op/s 2024-09-19T09:38:15.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:15 smithi038 ceph-mon[35271]: pgmap v506: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 772 op/s 2024-09-19T09:38:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:15 smithi028 ceph-mon[30972]: pgmap v506: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 772 op/s 2024-09-19T09:38:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:15 smithi078 ceph-mon[35619]: pgmap v506: 97 pgs: 97 active+clean; 410 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 772 op/s 2024-09-19T09:38:17.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:17 smithi038 ceph-mon[35271]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 781 op/s 2024-09-19T09:38:17.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:17 smithi028 ceph-mon[30972]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 781 op/s 2024-09-19T09:38:17.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:17 smithi078 ceph-mon[35619]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 781 op/s 2024-09-19T09:38:19.058 INFO:tasks.workunit.client.0.smithi028.stdout: 134 2082480 703 1519984 510 1138154 1768 2024-09-19T09:38:19.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:19 smithi038 ceph-mon[35271]: pgmap v508: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 779 op/s 2024-09-19T09:38:19.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:19 smithi028 ceph-mon[30972]: pgmap v508: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 779 op/s 2024-09-19T09:38:19.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:19 smithi078 ceph-mon[35619]: pgmap v508: 97 pgs: 97 active+clean; 416 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 779 op/s 2024-09-19T09:38:21.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:21 smithi038 ceph-mon[35271]: pgmap v509: 97 pgs: 97 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-09-19T09:38:21.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:21 smithi028 ceph-mon[30972]: pgmap v509: 97 pgs: 97 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-09-19T09:38:21.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:21 smithi078 ceph-mon[35619]: pgmap v509: 97 pgs: 97 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 592 op/s 2024-09-19T09:38:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:23 smithi038 ceph-mon[35271]: pgmap v510: 97 pgs: 97 active+clean; 418 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-09-19T09:38:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:23 smithi028 ceph-mon[30972]: pgmap v510: 97 pgs: 97 active+clean; 418 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-09-19T09:38:23.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:23 smithi078 ceph-mon[35619]: pgmap v510: 97 pgs: 97 active+clean; 418 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-09-19T09:38:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.163+0000 7f666781e640 1 -- 172.21.15.28:0/2824125054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 msgr2=0x7f6660109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.163+0000 7f666781e640 1 --2- 172.21.15.28:0/2824125054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f6660109670 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f665c009980 tx=0x7f665c02a4d0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.164+0000 7f666781e640 1 -- 172.21.15.28:0/2824125054 shutdown_connections 2024-09-19T09:38:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.164+0000 7f666781e640 1 --2- 172.21.15.28:0/2824125054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6660109bb0 0x7f6660119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.164+0000 7f666781e640 1 --2- 172.21.15.28:0/2824125054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f6660109670 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.164+0000 7f666781e640 1 --2- 172.21.15.28:0/2824125054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66600732d0 0x7f66600736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.164+0000 7f666781e640 1 -- 172.21.15.28:0/2824125054 >> 172.21.15.28:0/2824125054 conn(0x7f666006eb10 msgr2=0x7f6660070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.167+0000 7f666781e640 1 -- 172.21.15.28:0/2824125054 shutdown_connections 2024-09-19T09:38:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.167+0000 7f666781e640 1 -- 172.21.15.28:0/2824125054 wait complete. 2024-09-19T09:38:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 Processor -- start 2024-09-19T09:38:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 -- start start 2024-09-19T09:38:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f66601a7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6660109bb0 0x7f66601ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f666011c5b0 con 0x7f66601091f0 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f666011c430 con 0x7f6660109bb0 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.168+0000 7f666781e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f666011c730 con 0x7f66600732d0 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:55106/0 (socket says 172.21.15.28:55106) 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 -- 172.21.15.28:0/1794276214 learned_addr learned my addr 172.21.15.28:0/1794276214 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f6664db9640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f66601a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 -- 172.21.15.28:0/1794276214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6660109bb0 msgr2=0x7f66601ae180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6660109bb0 0x7f66601ae180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 -- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 msgr2=0x7f66601a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f66601a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f66655ba640 1 -- 172.21.15.28:0/1794276214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66601ae7f0 con 0x7f66600732d0 2024-09-19T09:38:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.169+0000 7f6664db9640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f66601a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:38:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.170+0000 7f66655ba640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f665800b9e0 tx=0x7f665800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.193+0000 7f66567fc640 1 -- 172.21.15.28:0/1794276214 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658019070 con 0x7f66600732d0 2024-09-19T09:38:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.193+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601aeae0 con 0x7f66600732d0 2024-09-19T09:38:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.193+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601af030 con 0x7f66600732d0 2024-09-19T09:38:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.193+0000 7f66567fc640 1 -- 172.21.15.28:0/1794276214 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665800cba0 con 0x7f66600732d0 2024-09-19T09:38:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.193+0000 7f66567fc640 1 -- 172.21.15.28:0/1794276214 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580126b0 con 0x7f66600732d0 2024-09-19T09:38:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.199+0000 7f66567fc640 1 -- 172.21.15.28:0/1794276214 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f665801c070 con 0x7f66600732d0 2024-09-19T09:38:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.200+0000 7f66567fc640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 0x7f664c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.200+0000 7f66567fc640 1 -- 172.21.15.28:0/1794276214 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f66580cee40 con 0x7f66600732d0 2024-09-19T09:38:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.201+0000 7f666a690740 1 Processor -- start 2024-09-19T09:38:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.201+0000 7f6664db9640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 0x7f664c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.201+0000 7f666a690740 1 -- start start 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fc0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 0x563e81fc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563e81f242f0 con 0x563e81fc0b70 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e81ecc270 con 0x563e81fc6540 2024-09-19T09:38:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f666a690740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e81f202b0 con 0x563e81fc14b0 2024-09-19T09:38:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f6665dbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f66655ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fc0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f6665dbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:49794/0 (socket says 172.21.15.28:49794) 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f6665dbb640 1 -- 172.21.15.28:0/1129126303 learned_addr learned my addr 172.21.15.28:0/1129126303 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.202+0000 7f6664db9640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 0x7f664c0a5920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f66601a80e0 tx=0x7f665c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6664db9640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 0x563e81fc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 -- 172.21.15.28:0/1129126303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 msgr2=0x563e81fc5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 0x563e81fc5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 -- 172.21.15.28:0/1129126303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 msgr2=0x563e81fc0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fc0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 -- 172.21.15.28:0/1129126303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e81ecb340 con 0x563e81fc6540 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f66655ba640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fc0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6664db9640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 0x563e81fc5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:38:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.203+0000 7f6665dbb640 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66500096f0 tx=0x7f6650009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84109652a9ec418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.227+0000 7f664b7fe640 1 -- 172.21.15.28:0/1129126303 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650015020 con 0x563e81fc6540 2024-09-19T09:38:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.228+0000 7f664b7fe640 1 -- 172.21.15.28:0/1129126303 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665000bdb0 con 0x563e81fc6540 2024-09-19T09:38:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.229+0000 7f664b7fe640 1 -- 172.21.15.28:0/1129126303 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650010380 con 0x563e81fc6540 2024-09-19T09:38:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.231+0000 7f666a690740 1 -- 172.21.15.28:0/1129126303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 msgr2=0x563e81fc8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.231+0000 7f666a690740 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66500096f0 tx=0x7f6650009bb0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.233+0000 7f666a690740 1 -- 172.21.15.28:0/1129126303 shutdown_connections 2024-09-19T09:38:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.233+0000 7f666a690740 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fc8a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.233+0000 7f666a690740 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc14b0 0x563e81fc5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.233+0000 7f666a690740 1 --2- 172.21.15.28:0/1129126303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fc0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.233+0000 7f666a690740 1 -- 172.21.15.28:0/1129126303 >> 172.21.15.28:0/1129126303 conn(0x563e81fbc1e0 msgr2=0x563e81fbc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.236+0000 7f666a690740 1 -- 172.21.15.28:0/1129126303 shutdown_connections 2024-09-19T09:38:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.236+0000 7f666a690740 1 -- 172.21.15.28:0/1129126303 wait complete. 2024-09-19T09:38:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.237+0000 7f666a690740 1 -- start start 2024-09-19T09:38:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.237+0000 7f666a690740 1 Processor -- start 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563e81fc9510 con 0x563e81fc14b0 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e81ead290 con 0x563e81fc6540 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e81fc9690 con 0x563e81fc0b70 2024-09-19T09:38:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f666a690740 10 client.0 _ll_register_callbacks cb 0x563e81ede140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:55122/0 (socket says 172.21.15.28:55122) 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f66655ba640 1 -- 172.21.15.28:0/1460255809 learned_addr learned my addr 172.21.15.28:0/1460255809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f6664db9640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.238+0000 7f6665dbb640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 -- 172.21.15.28:0/1460255809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 msgr2=0x563e81fd2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 -- 172.21.15.28:0/1460255809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 msgr2=0x563e81fcb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fcb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 -- 172.21.15.28:0/1460255809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e81c37210 con 0x563e81fc0b70 2024-09-19T09:38:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f6664db9640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:38:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.239+0000 7f66655ba640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6658098210 tx=0x7f66580ccb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b46b69191dfa560 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f6648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:38:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658014bc0 con 0x563e81fc0b70 2024-09-19T09:38:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665000bdb0 con 0x563e81fc0b70 2024-09-19T09:38:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6658098480 con 0x563e81fc0b70 2024-09-19T09:38:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f665809b3f0 con 0x563e81fc0b70 2024-09-19T09:38:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f666a690740 10 client.26221 fetch_fsmap learned FSMap version 36 2024-09-19T09:38:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.266+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563e81d03620 con 0x563e81fc0b70 2024-09-19T09:38:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.267+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665000bdb0 con 0x563e81fc0b70 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f66580bb1c0 con 0x563e81fc0b70 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 10 client.26221 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 20 client.26221 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e81fd6e10 0x563e81fd9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 4 client.26221 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:38:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e81f31a20 con 0x563e81fd6e10 2024-09-19T09:38:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.268+0000 7f6664db9640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e81fd6e10 0x563e81fd9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.269+0000 7f6664db9640 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e81fd6e10 0x563e81fd9290 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccd6c96d1f60b37 server_cookie=b86356a8a212522e in_seq=0 out_seq=0 2024-09-19T09:38:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.271+0000 7f6648ff9640 10 client.26221 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.273+0000 7f6648ff9640 1 -- 172.21.15.28:0/1460255809 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e81f31a20 con 0x563e81fd6e10 2024-09-19T09:38:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.273+0000 7f6648ff9640 10 client.26221 handle_command_reply: tid=0 2024-09-19T09:38:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.273+0000 7f666a690740 1 client.26221 shutdown 2024-09-19T09:38:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.278+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 msgr2=0x563e81fcad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.278+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6658098210 tx=0x7f66580ccb80 comp rx=0 tx=0).stop 2024-09-19T09:38:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 shutdown_connections 2024-09-19T09:38:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e81fd6e10 0x563e81fd9290 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 --2- 172.21.15.28:0/1460255809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fcad90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.280+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 >> 172.21.15.28:0/1460255809 conn(0x563e81fbc1e0 msgr2=0x563e81fbd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.283+0000 7f6648ff9640 0 client.26221 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.283+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 shutdown_connections 2024-09-19T09:38:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.283+0000 7f666a690740 1 -- 172.21.15.28:0/1460255809 wait complete. 2024-09-19T09:38:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.283+0000 7f666a690740 20 client.26221 trim_cache size 0 max 16384 2024-09-19T09:38:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.498+0000 7f666a690740 1 Processor -- start 2024-09-19T09:38:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 -- start start 2024-09-19T09:38:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fc2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f664400e480 con 0x563e81fc14b0 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6644010020 con 0x563e81fc6540 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f666a690740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f664400eb50 con 0x563e81fc0b70 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f66655ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:55136/0 (socket says 172.21.15.28:55136) 2024-09-19T09:38:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.500+0000 7f66655ba640 1 -- 172.21.15.28:0/3200954687 learned_addr learned my addr 172.21.15.28:0/3200954687 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f6664db9640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fc2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f6665dbb640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 -- 172.21.15.28:0/3200954687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 msgr2=0x563e81fd22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 -- 172.21.15.28:0/3200954687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 msgr2=0x563e81fc2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fc2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 -- 172.21.15.28:0/3200954687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e81fca8d0 con 0x563e81fc0b70 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f6664db9640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fc2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:38:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.501+0000 7f66655ba640 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66580094d0 tx=0x7f6658014920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5746c021eec64ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.524+0000 7f664affd640 1 -- 172.21.15.28:0/3200954687 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658014bc0 con 0x563e81fc0b70 2024-09-19T09:38:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.524+0000 7f664affd640 1 -- 172.21.15.28:0/3200954687 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66580a4180 con 0x563e81fc0b70 2024-09-19T09:38:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.528+0000 7f664affd640 1 -- 172.21.15.28:0/3200954687 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580b4260 con 0x563e81fc0b70 2024-09-19T09:38:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 -- 172.21.15.28:0/3200954687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 msgr2=0x563e81fc2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66580094d0 tx=0x7f6658014920 comp rx=0 tx=0).stop 2024-09-19T09:38:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 -- 172.21.15.28:0/3200954687 shutdown_connections 2024-09-19T09:38:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc6540 0x563e81fd22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc14b0 0x563e81fc2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 --2- 172.21.15.28:0/3200954687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc0b70 0x563e81fc2650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.529+0000 7f666a690740 1 -- 172.21.15.28:0/3200954687 >> 172.21.15.28:0/3200954687 conn(0x563e81fbc1e0 msgr2=0x563e81fbc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.531+0000 7f666a690740 1 -- 172.21.15.28:0/3200954687 shutdown_connections 2024-09-19T09:38:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.531+0000 7f666a690740 1 -- 172.21.15.28:0/3200954687 wait complete. 2024-09-19T09:38:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.532+0000 7f666a690740 1 -- start start 2024-09-19T09:38:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.532+0000 7f666a690740 1 Processor -- start 2024-09-19T09:38:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.533+0000 7f666a690740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 0x563e81fce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f66655ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f66655ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45060/0 (socket says 172.21.15.28:45060) 2024-09-19T09:38:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 0x563e81fcee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f664400f950 con 0x563e81fc0b70 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f664400f220 con 0x563e81fc14b0 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f66655ba640 1 -- 172.21.15.28:0/3392842603 learned_addr learned my addr 172.21.15.28:0/3392842603 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f664400c460 con 0x563e81fc6540 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f666a690740 10 client.0 _ll_register_callbacks cb 0x7f6658098850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f6664db9640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 0x563e81fce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.534+0000 7f6665dbb640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 0x563e81fcee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 -- 172.21.15.28:0/3392842603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 msgr2=0x563e81fce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 0x563e81fce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 -- 172.21.15.28:0/3392842603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 msgr2=0x563e81fd1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e81fd2430 con 0x563e81fc6540 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6664db9640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 0x563e81fce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f66655ba640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.535+0000 7f6665dbb640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 0x563e81fcee50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f66500096c0 tx=0x7f665000bbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51289231388aaf02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.553+0000 7f6648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.553+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650015040 con 0x563e81fc6540 2024-09-19T09:38:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.553+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66580a4180 con 0x563e81fc6540 2024-09-19T09:38:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.555+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6650004040 con 0x563e81fc6540 2024-09-19T09:38:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.555+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f665002a050 con 0x563e81fc6540 2024-09-19T09:38:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.555+0000 7f666a690740 10 client.26227 fetch_fsmap learned FSMap version 36 2024-09-19T09:38:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.555+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563e82040a10 con 0x563e81fc6540 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665001e070 con 0x563e81fc6540 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f6650023450 con 0x563e81fc6540 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 10 client.26227 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 10 client.26227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 20 client.26227 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e820412f0 0x563e82043710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 4 client.26227 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:38:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f66580a4180 con 0x563e820412f0 2024-09-19T09:38:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.558+0000 7f66655ba640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e820412f0 0x563e82043710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.559+0000 7f6648ff9640 10 client.26227 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:25.559+0000 7f66655ba640 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e820412f0 0x563e82043710 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d8ea9eb2c6b88d server_cookie=35f2c1ad51487f4a in_seq=0 out_seq=0 2024-09-19T09:38:25.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:25 smithi038 ceph-mon[35271]: pgmap v511: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-19T09:38:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:25 smithi028 ceph-mon[30972]: pgmap v511: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-19T09:38:25.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:25 smithi078 ceph-mon[35619]: pgmap v511: 97 pgs: 97 active+clean; 423 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-19T09:38:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:27 smithi038 ceph-mon[35271]: pgmap v512: 97 pgs: 97 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-19T09:38:27.771 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:27 smithi028 ceph-mon[30972]: pgmap v512: 97 pgs: 97 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-19T09:38:27.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:27 smithi078 ceph-mon[35619]: pgmap v512: 97 pgs: 97 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-19T09:38:29.058 INFO:tasks.workunit.client.0.smithi028.stdout: 140 1716130 399 1246638 370 926438 975 2024-09-19T09:38:29.675 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:29 smithi038 ceph-mon[35271]: pgmap v513: 97 pgs: 97 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-19T09:38:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:29 smithi028 ceph-mon[30972]: pgmap v513: 97 pgs: 97 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-19T09:38:29.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:29 smithi078 ceph-mon[35619]: pgmap v513: 97 pgs: 97 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-19T09:38:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.472+0000 7f6648ff9640 1 -- 172.21.15.28:0/3392842603 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f66580a4180 con 0x563e820412f0 2024-09-19T09:38:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.472+0000 7f6648ff9640 10 client.26227 handle_command_reply: tid=0 2024-09-19T09:38:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.472+0000 7f666a690740 1 client.26227 shutdown 2024-09-19T09:38:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 msgr2=0x563e81fcee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 0x563e81fcee50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f66500096c0 tx=0x7f665000bbd0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 shutdown_connections 2024-09-19T09:38:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e81fc14b0 0x563e81fce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x563e820412f0 0x563e82043710 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e81fc6540 0x563e81fcee50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 --2- 172.21.15.28:0/3392842603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563e81fc0b70 0x563e81fd1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 >> 172.21.15.28:0/3392842603 conn(0x563e81fbc1e0 msgr2=0x563e81fc2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f6648ff9640 0 client.26227 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 shutdown_connections 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.479+0000 7f666a690740 1 -- 172.21.15.28:0/3392842603 wait complete. 2024-09-19T09:38:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.480+0000 7f666a690740 20 client.26227 trim_cache size 0 max 16384 2024-09-19T09:38:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:38:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:38:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:38:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:38:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.484+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 msgr2=0x7f664c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.484+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 0x7f664c0a5920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f66601a80e0 tx=0x7f665c031400 comp rx=0 tx=0).stop 2024-09-19T09:38:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.485+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 msgr2=0x7f66601a6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.485+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f665800b9e0 tx=0x7f665800bea0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 shutdown_connections 2024-09-19T09:38:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f664c0a3460 0x7f664c0a5920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6660109bb0 0x7f66601ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66601091f0 0x7f66601a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 --2- 172.21.15.28:0/1794276214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66600732d0 0x7f66601a6bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.486+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 >> 172.21.15.28:0/1794276214 conn(0x7f666006eb10 msgr2=0x7f6660071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.487+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 shutdown_connections 2024-09-19T09:38:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:30.488+0000 7f666781e640 1 -- 172.21.15.28:0/1794276214 wait complete. 2024-09-19T09:38:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:31 smithi028 ceph-mon[30972]: pgmap v514: 97 pgs: 97 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 726 op/s 2024-09-19T09:38:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:31 smithi078 ceph-mon[35619]: pgmap v514: 97 pgs: 97 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 726 op/s 2024-09-19T09:38:31.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:31 smithi038 ceph-mon[35271]: pgmap v514: 97 pgs: 97 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 726 op/s 2024-09-19T09:38:32.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:32 smithi028 ceph-mon[30972]: pgmap v515: 97 pgs: 97 active+clean; 483 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 922 op/s 2024-09-19T09:38:32.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:32 smithi078 ceph-mon[35619]: pgmap v515: 97 pgs: 97 active+clean; 483 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 922 op/s 2024-09-19T09:38:32.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:32 smithi038 ceph-mon[35271]: pgmap v515: 97 pgs: 97 active+clean; 483 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 922 op/s 2024-09-19T09:38:34.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:34 smithi078 ceph-mon[35619]: pgmap v516: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 1.63k op/s 2024-09-19T09:38:34.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:34 smithi038 ceph-mon[35271]: pgmap v516: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 1.63k op/s 2024-09-19T09:38:35.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:34 smithi028 ceph-mon[30972]: pgmap v516: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 1.63k op/s 2024-09-19T09:38:36.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:36 smithi078 ceph-mon[35619]: pgmap v517: 97 pgs: 97 active+clean; 468 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.6 MiB/s wr, 1.69k op/s 2024-09-19T09:38:36.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:36 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:36 smithi038 ceph-mon[35271]: pgmap v517: 97 pgs: 97 active+clean; 468 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.6 MiB/s wr, 1.69k op/s 2024-09-19T09:38:36.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:36 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:36 smithi028 ceph-mon[30972]: pgmap v517: 97 pgs: 97 active+clean; 468 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.6 MiB/s wr, 1.69k op/s 2024-09-19T09:38:37.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:36 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:38:38.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:38 smithi038 ceph-mon[35271]: pgmap v518: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 1.68k op/s 2024-09-19T09:38:39.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:38 smithi028 ceph-mon[30972]: pgmap v518: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 1.68k op/s 2024-09-19T09:38:39.058 INFO:tasks.workunit.client.0.smithi028.stdout: 149 1667784 472 1199409 427 896995 1228 2024-09-19T09:38:39.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:38 smithi078 ceph-mon[35619]: pgmap v518: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 1.68k op/s 2024-09-19T09:38:40.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:40 smithi038 ceph-mon[35271]: pgmap v519: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 1.61k op/s 2024-09-19T09:38:41.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:40 smithi028 ceph-mon[30972]: pgmap v519: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 1.61k op/s 2024-09-19T09:38:41.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:40 smithi078 ceph-mon[35619]: pgmap v519: 97 pgs: 97 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 1.61k op/s 2024-09-19T09:38:42.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:42 smithi038 ceph-mon[35271]: pgmap v520: 97 pgs: 97 active+clean; 472 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 1.11k op/s 2024-09-19T09:38:43.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:42 smithi028 ceph-mon[30972]: pgmap v520: 97 pgs: 97 active+clean; 472 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 1.11k op/s 2024-09-19T09:38:43.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:42 smithi078 ceph-mon[35619]: pgmap v520: 97 pgs: 97 active+clean; 472 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 1.11k op/s 2024-09-19T09:38:44.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:44 smithi038 ceph-mon[35271]: pgmap v521: 97 pgs: 97 active+clean; 476 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 920 op/s 2024-09-19T09:38:45.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:44 smithi028 ceph-mon[30972]: pgmap v521: 97 pgs: 97 active+clean; 476 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 920 op/s 2024-09-19T09:38:45.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:44 smithi078 ceph-mon[35619]: pgmap v521: 97 pgs: 97 active+clean; 476 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 920 op/s 2024-09-19T09:38:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.314+0000 7f6e75e58640 1 -- 172.21.15.28:0/1589636098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010b4e0 msgr2=0x7f6e7010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.314+0000 7f6e75e58640 1 --2- 172.21.15.28:0/1589636098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010b4e0 0x7f6e7010d950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009900 tx=0x7f6e6002a420 comp rx=0 tx=0).stop 2024-09-19T09:38:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 -- 172.21.15.28:0/1589636098 shutdown_connections 2024-09-19T09:38:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 --2- 172.21.15.28:0/1589636098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e7010de90 0x7f6e70110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 --2- 172.21.15.28:0/1589636098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010b4e0 0x7f6e7010d950 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 --2- 172.21.15.28:0/1589636098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e70073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 -- 172.21.15.28:0/1589636098 >> 172.21.15.28:0/1589636098 conn(0x7f6e7006eb10 msgr2=0x7f6e70070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.319+0000 7f6e75e58640 1 -- 172.21.15.28:0/1589636098 shutdown_connections 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.320+0000 7f6e75e58640 1 -- 172.21.15.28:0/1589636098 wait complete. 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.320+0000 7f6e75e58640 1 Processor -- start 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 -- start start 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e7010b4e0 0x7f6e701a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 0x7f6e7007a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7011c430 con 0x7f6e7010de90 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7011c2b0 con 0x7f6e7010b4e0 2024-09-19T09:38:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e75e58640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7011c5b0 con 0x7f6e70072d80 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:47536/0 (socket says 172.21.15.28:47536) 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.321+0000 7f6e74e56640 1 -- 172.21.15.28:0/3205073466 learned_addr learned my addr 172.21.15.28:0/3205073466 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e74e56640 1 -- 172.21.15.28:0/3205073466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e7010b4e0 msgr2=0x7f6e701a6f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e74e56640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e7010b4e0 0x7f6e701a6f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e74e56640 1 -- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 msgr2=0x7f6e7007a450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e75657640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 0x7f6e7007a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e74e56640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 0x7f6e7007a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e74e56640 1 -- 172.21.15.28:0/3205073466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7007aaf0 con 0x7f6e70072d80 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.322+0000 7f6e75657640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 0x7f6e7007a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:38:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.323+0000 7f6e74e56640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c890 tx=0x7f6e6400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.357+0000 7f6e6dffb640 1 -- 172.21.15.28:0/3205073466 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64004220 con 0x7f6e70072d80 2024-09-19T09:38:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.357+0000 7f6e6dffb640 1 -- 172.21.15.28:0/3205073466 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e640043c0 con 0x7f6e70072d80 2024-09-19T09:38:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.357+0000 7f6e6dffb640 1 -- 172.21.15.28:0/3205073466 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e640049e0 con 0x7f6e70072d80 2024-09-19T09:38:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.357+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7007ade0 con 0x7f6e70072d80 2024-09-19T09:38:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.357+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7007b2e0 con 0x7f6e70072d80 2024-09-19T09:38:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.361+0000 7f6e79e59740 1 Processor -- start 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 -- start start 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45541b70 0x557c45541f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c455424b0 0x557c45546f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557c454a52f0 con 0x557c45547540 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c4544d270 con 0x557c45541b70 2024-09-19T09:38:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.362+0000 7f6e79e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c454a12b0 con 0x557c455424b0 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55720/0 (socket says 172.21.15.28:55720) 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 -- 172.21.15.28:0/4205218707 learned_addr learned my addr 172.21.15.28:0/4205218707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 -- 172.21.15.28:0/4205218707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c455424b0 msgr2=0x557c45546f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c455424b0 0x557c45546f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 -- 172.21.15.28:0/4205218707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45541b70 msgr2=0x557c45541f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:38:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45541b70 0x557c45541f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.363+0000 7f6e6ffff640 1 -- 172.21.15.28:0/4205218707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4544c340 con 0x557c45547540 2024-09-19T09:38:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.364+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6e60031470 tx=0x7f6e60005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baef7d90969145e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.368+0000 7f6e6dffb640 1 -- 172.21.15.28:0/3205073466 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6e64004c40 con 0x7f6e70072d80 2024-09-19T09:38:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.370+0000 7f6e6dffb640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 0x7f6e580a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.370+0000 7f6e75657640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 0x7f6e580a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.371+0000 7f6e75657640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 0x7f6e580a5920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6e68005fd0 tx=0x7f6e680093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.383+0000 7f6e6dffb640 1 -- 172.21.15.28:0/3205073466 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6e640d0930 con 0x7f6e70072d80 2024-09-19T09:38:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.403+0000 7f6e5effd640 1 -- 172.21.15.28:0/4205218707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60040070 con 0x557c45547540 2024-09-19T09:38:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.403+0000 7f6e5effd640 1 -- 172.21.15.28:0/4205218707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e60031640 con 0x557c45547540 2024-09-19T09:38:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.404+0000 7f6e79e59740 1 -- 172.21.15.28:0/4205218707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 msgr2=0x557c45549a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.404+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6e60031470 tx=0x7f6e60005780 comp rx=0 tx=0).stop 2024-09-19T09:38:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.407+0000 7f6e79e59740 1 -- 172.21.15.28:0/4205218707 shutdown_connections 2024-09-19T09:38:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.407+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45549a00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.407+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c455424b0 0x557c45546f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.407+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4205218707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45541b70 0x557c45541f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.407+0000 7f6e79e59740 1 -- 172.21.15.28:0/4205218707 >> 172.21.15.28:0/4205218707 conn(0x557c4553d1e0 msgr2=0x557c4553d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.415+0000 7f6e79e59740 1 -- 172.21.15.28:0/4205218707 shutdown_connections 2024-09-19T09:38:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.415+0000 7f6e79e59740 1 -- 172.21.15.28:0/4205218707 wait complete. 2024-09-19T09:38:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.415+0000 7f6e79e59740 1 -- start start 2024-09-19T09:38:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.415+0000 7f6e79e59740 1 Processor -- start 2024-09-19T09:38:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45553260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557c4554a500 con 0x557c45547540 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c4542e290 con 0x557c455424b0 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c4554a680 con 0x557c45541b70 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.416+0000 7f6e79e59740 10 client.0 _ll_register_callbacks cb 0x557c4545f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e75657640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45553260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:58284/0 (socket says 172.21.15.28:58284) 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 -- 172.21.15.28:0/3142033452 learned_addr learned my addr 172.21.15.28:0/3142033452 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 -- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 msgr2=0x557c4554bce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554bce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.417+0000 7f6e6ffff640 1 -- 172.21.15.28:0/3142033452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 msgr2=0x557c45553260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.418+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45553260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.418+0000 7f6e6ffff640 1 -- 172.21.15.28:0/3142033452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c451b8210 con 0x557c455424b0 2024-09-19T09:38:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.418+0000 7f6e75657640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45553260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.418+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e60031a50 tx=0x7f6e600036c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f75031718a5245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.455+0000 7f6e53fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:38:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.455+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60040050 con 0x557c455424b0 2024-09-19T09:38:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.455+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6e60003fc0 con 0x557c455424b0 2024-09-19T09:38:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.455+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e60031640 con 0x557c455424b0 2024-09-19T09:38:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.461+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e60055070 con 0x557c455424b0 2024-09-19T09:38:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.462+0000 7f6e79e59740 10 client.26246 fetch_fsmap learned FSMap version 37 2024-09-19T09:38:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.462+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c45284620 con 0x557c455424b0 2024-09-19T09:38:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.482+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e60031640 con 0x557c455424b0 2024-09-19T09:38:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f6e60045450 con 0x557c455424b0 2024-09-19T09:38:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 10 client.26246 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:38:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 20 client.26246 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c45568ab0 0x557c4556aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 4 client.26246 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.483+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c454b2a20 con 0x557c45568ab0 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.484+0000 7f6e74e56640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c45568ab0 0x557c4556aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.484+0000 7f6e74e56640 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c45568ab0 0x557c4556aef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4c8507e0335515 server_cookie=a1686613146bed3f in_seq=0 out_seq=0 2024-09-19T09:38:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.484+0000 7f6e53fff640 10 client.26246 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.488+0000 7f6e53fff640 1 -- 172.21.15.28:0/3142033452 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c454b2a20 con 0x557c45568ab0 2024-09-19T09:38:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e53fff640 10 client.26246 handle_command_reply: tid=0 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 client.26246 shutdown 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 msgr2=0x557c4554c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e60031a50 tx=0x7f6e600036c0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 shutdown_connections 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c455424b0 0x557c4554c220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c45568ab0 0x557c4556aef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c45547540 0x557c45553260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 --2- 172.21.15.28:0/3142033452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 >> 172.21.15.28:0/3142033452 conn(0x557c4553d1e0 msgr2=0x557c4553e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.489+0000 7f6e53fff640 0 client.26246 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.490+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 shutdown_connections 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.490+0000 7f6e79e59740 1 -- 172.21.15.28:0/3142033452 wait complete. 2024-09-19T09:38:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.490+0000 7f6e79e59740 20 client.26246 trim_cache size 0 max 16384 2024-09-19T09:38:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.674+0000 7f6e79e59740 1 Processor -- start 2024-09-19T09:38:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.682+0000 7f6e79e59740 1 -- start start 2024-09-19T09:38:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.682+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.683+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.683+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:47540/0 (socket says 172.21.15.28:47540) 2024-09-19T09:38:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.683+0000 7f6e74e56640 1 -- 172.21.15.28:0/622725703 learned_addr learned my addr 172.21.15.28:0/622725703 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.688+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c45553220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.688+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4553dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.688+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c45553220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.688+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400e360 con 0x557c455424b0 2024-09-19T09:38:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.688+0000 7f6e75657640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4553dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400fed0 con 0x557c45547540 2024-09-19T09:38:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400ea30 con 0x557c45541b70 2024-09-19T09:38:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e74e56640 1 -- 172.21.15.28:0/622725703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 msgr2=0x557c4553dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e74e56640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4553dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e74e56640 1 -- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 msgr2=0x557c45553220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e74e56640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c45553220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e74e56640 1 -- 172.21.15.28:0/622725703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4554a680 con 0x557c45541b70 2024-09-19T09:38:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.692+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c45553220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:38:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.693+0000 7f6e74e56640 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e64099970 tx=0x7f6e64005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210d7cfbf4db7866 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.704+0000 7f6e5e7fc640 1 -- 172.21.15.28:0/622725703 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6401d040 con 0x557c45541b70 2024-09-19T09:38:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.705+0000 7f6e5e7fc640 1 -- 172.21.15.28:0/622725703 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e6409cfa0 con 0x557c45541b70 2024-09-19T09:38:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.705+0000 7f6e5e7fc640 1 -- 172.21.15.28:0/622725703 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64004f30 con 0x557c45541b70 2024-09-19T09:38:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.707+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 msgr2=0x557c4554c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.707+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e64099970 tx=0x7f6e64005300 comp rx=0 tx=0).stop 2024-09-19T09:38:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.709+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 shutdown_connections 2024-09-19T09:38:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.709+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4553dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.709+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c45553220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.709+0000 7f6e79e59740 1 --2- 172.21.15.28:0/622725703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c4554c1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.709+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 >> 172.21.15.28:0/622725703 conn(0x557c4553d1e0 msgr2=0x557c4554ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.710+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 shutdown_connections 2024-09-19T09:38:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.710+0000 7f6e79e59740 1 -- 172.21.15.28:0/622725703 wait complete. 2024-09-19T09:38:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.710+0000 7f6e79e59740 1 -- start start 2024-09-19T09:38:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.711+0000 7f6e79e59740 1 Processor -- start 2024-09-19T09:38:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c4554fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e74e56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:47546/0 (socket says 172.21.15.28:47546) 2024-09-19T09:38:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e74e56640 1 -- 172.21.15.28:0/4282339352 learned_addr learned my addr 172.21.15.28:0/4282339352 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:38:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4554fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400f800 con 0x557c455424b0 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c4554fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400f100 con 0x557c45547540 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5400c3f0 con 0x557c45541b70 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e79e59740 10 client.0 _ll_register_callbacks cb 0x7f6e600056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.712+0000 7f6e75657640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4554fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 -- 172.21.15.28:0/4282339352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 msgr2=0x557c4554fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4554fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 -- 172.21.15.28:0/4282339352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 msgr2=0x557c4554fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:38:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c4554fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c455d1780 con 0x557c45541b70 2024-09-19T09:38:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c4554fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:38:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.713+0000 7f6e74e56640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e64013040 tx=0x7f6e640a1350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7296e82258a88e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:38:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.728+0000 7f6e53fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:38:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.729+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a9060 con 0x557c45541b70 2024-09-19T09:38:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.729+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6e640a5780 con 0x557c45541b70 2024-09-19T09:38:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.729+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e6409cfa0 con 0x557c45541b70 2024-09-19T09:38:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.730+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e640a5e60 con 0x557c45541b70 2024-09-19T09:38:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.730+0000 7f6e79e59740 10 client.26236 fetch_fsmap learned FSMap version 37 2024-09-19T09:38:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.730+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c455d2950 con 0x557c45541b70 2024-09-19T09:38:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.730+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e640a5980 con 0x557c45541b70 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f6e6409be60 con 0x557c45541b70 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 10 client.26236 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 20 client.26236 populate_metadata read hostname 'smithi028' 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c455d31c0 0x557c455d55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 4 client.26236 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:38:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6e6409cfa0 con 0x557c455d31c0 2024-09-19T09:38:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.731+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c455d31c0 0x557c455d55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:38:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.732+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c455d31c0 0x557c455d55e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e2d434502bb1ff server_cookie=546370464733e50a in_seq=0 out_seq=0 2024-09-19T09:38:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:45.733+0000 7f6e53fff640 10 client.26236 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:46.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:46 smithi038 ceph-mon[35271]: pgmap v522: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T09:38:47.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:46 smithi028 ceph-mon[30972]: pgmap v522: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T09:38:47.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:46 smithi078 ceph-mon[35619]: pgmap v522: 97 pgs: 97 active+clean; 478 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-09-19T09:38:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:48 smithi038 ceph-mon[35271]: pgmap v523: 97 pgs: 97 active+clean; 485 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-19T09:38:49.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:48 smithi028 ceph-mon[30972]: pgmap v523: 97 pgs: 97 active+clean; 485 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-19T09:38:49.058 INFO:tasks.workunit.client.0.smithi028.stdout: 157 2086184 459 1507079 469 1109400 1723 2024-09-19T09:38:49.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:48 smithi078 ceph-mon[35619]: pgmap v523: 97 pgs: 97 active+clean; 485 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-19T09:38:50.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:50 smithi038 ceph-mon[35271]: pgmap v524: 97 pgs: 97 active+clean; 483 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-19T09:38:51.019 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:50 smithi028 ceph-mon[30972]: pgmap v524: 97 pgs: 97 active+clean; 483 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-19T09:38:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:50 smithi078 ceph-mon[35619]: pgmap v524: 97 pgs: 97 active+clean; 483 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-09-19T09:38:52.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:52 smithi038 ceph-mon[35271]: pgmap v525: 97 pgs: 97 active+clean; 487 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-19T09:38:53.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:52 smithi028 ceph-mon[30972]: pgmap v525: 97 pgs: 97 active+clean; 487 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-19T09:38:53.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:52 smithi078 ceph-mon[35619]: pgmap v525: 97 pgs: 97 active+clean; 487 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e53fff640 1 -- 172.21.15.28:0/4282339352 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6e6409cfa0 con 0x557c455d31c0 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e53fff640 10 client.26236 handle_command_reply: tid=0 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 client.26236 shutdown 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 msgr2=0x557c45553010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e64013040 tx=0x7f6e640a1350 comp rx=0 tx=0).stop 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 shutdown_connections 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c45547540 0x557c4554fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x557c455d31c0 0x557c455d55e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557c455424b0 0x557c4554fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 --2- 172.21.15.28:0/4282339352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c45541b70 0x557c45553010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e53fff640 0 client.26236 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 >> 172.21.15.28:0/4282339352 conn(0x557c4553d1e0 msgr2=0x557c4553ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 shutdown_connections 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.506+0000 7f6e79e59740 1 -- 172.21.15.28:0/4282339352 wait complete. 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e79e59740 20 client.26236 trim_cache size 0 max 16384 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:38:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:38:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:38:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 msgr2=0x7f6e580a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 0x7f6e580a5920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6e68005fd0 tx=0x7f6e680093c0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 msgr2=0x7f6e701a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c890 tx=0x7f6e6400cd50 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 shutdown_connections 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6e580a3460 0x7f6e580a5920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e7010b4e0 0x7f6e701a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e7010de90 0x7f6e7007a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 --2- 172.21.15.28:0/3205073466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e70072d80 0x7f6e701a69c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 >> 172.21.15.28:0/3205073466 conn(0x7f6e7006eb10 msgr2=0x7f6e7010ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 shutdown_connections 2024-09-19T09:38:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:38:54.507+0000 7f6e75e58640 1 -- 172.21.15.28:0/3205073466 wait complete. 2024-09-19T09:38:54.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:54 smithi038 ceph-mon[35271]: pgmap v526: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 971 op/s 2024-09-19T09:38:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:54 smithi028 ceph-mon[30972]: pgmap v526: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 971 op/s 2024-09-19T09:38:55.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:54 smithi078 ceph-mon[35619]: pgmap v526: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 971 op/s 2024-09-19T09:38:56.928 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:56 smithi038 ceph-mon[35271]: pgmap v527: 97 pgs: 97 active+clean; 537 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 1.29k op/s 2024-09-19T09:38:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:56 smithi028 ceph-mon[30972]: pgmap v527: 97 pgs: 97 active+clean; 537 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 1.29k op/s 2024-09-19T09:38:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:56 smithi078 ceph-mon[35619]: pgmap v527: 97 pgs: 97 active+clean; 537 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 1.29k op/s 2024-09-19T09:38:58.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:58 smithi038 ceph-mon[35271]: pgmap v528: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.41k op/s 2024-09-19T09:38:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:58 smithi028 ceph-mon[30972]: pgmap v528: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.41k op/s 2024-09-19T09:38:59.058 INFO:tasks.workunit.client.0.smithi028.stdout: 160 973690 257 712927 269 516292 753 2024-09-19T09:38:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:58 smithi078 ceph-mon[35619]: pgmap v528: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.41k op/s 2024-09-19T09:38:59.918 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:38:59 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:38:59.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:38:59 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:39:00.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:38:59 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:39:01.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:01 smithi078 ceph-mon[35619]: pgmap v529: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 1.47k op/s 2024-09-19T09:39:01.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:01 smithi038 ceph-mon[35271]: pgmap v529: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 1.47k op/s 2024-09-19T09:39:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:01 smithi028 ceph-mon[30972]: pgmap v529: 97 pgs: 97 active+clean; 529 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 1.47k op/s 2024-09-19T09:39:03.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:03 smithi078 ceph-mon[35619]: pgmap v530: 97 pgs: 97 active+clean; 535 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 1.46k op/s 2024-09-19T09:39:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:03 smithi038 ceph-mon[35271]: pgmap v530: 97 pgs: 97 active+clean; 535 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 1.46k op/s 2024-09-19T09:39:03.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:03 smithi028 ceph-mon[30972]: pgmap v530: 97 pgs: 97 active+clean; 535 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 1.46k op/s 2024-09-19T09:39:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: pgmap v531: 97 pgs: 97 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 1.39k op/s 2024-09-19T09:39:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi028 to 177.0M 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: Unable to set osd_memory_target on smithi028 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:39:05.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: pgmap v531: 97 pgs: 97 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 1.39k op/s 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi028 to 177.0M 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: Unable to set osd_memory_target on smithi028 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:39:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:39:05.677 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:39:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: pgmap v531: 97 pgs: 97 active+clean; 533 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 1.39k op/s 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi028 to 177.0M 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: Unable to set osd_memory_target on smithi028 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:39:05.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:39:07.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:07 smithi078 ceph-mon[35619]: pgmap v532: 97 pgs: 97 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 756 op/s 2024-09-19T09:39:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:07 smithi038 ceph-mon[35271]: pgmap v532: 97 pgs: 97 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 756 op/s 2024-09-19T09:39:07.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:07 smithi028 ceph-mon[30972]: pgmap v532: 97 pgs: 97 active+clean; 532 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 756 op/s 2024-09-19T09:39:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.855+0000 7f476bfff640 1 -- 172.21.15.28:0/1193772932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10b4e0 msgr2=0x7f476c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.855+0000 7f476bfff640 1 --2- 172.21.15.28:0/1193772932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10b4e0 0x7f476c10d950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4764009cd0 tx=0x7f476402a930 comp rx=0 tx=0).stop 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 -- 172.21.15.28:0/1193772932 shutdown_connections 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 --2- 172.21.15.28:0/1193772932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c10de90 0x7f476c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 --2- 172.21.15.28:0/1193772932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10b4e0 0x7f476c10d950 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 --2- 172.21.15.28:0/1193772932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c072d80 0x7f476c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 -- 172.21.15.28:0/1193772932 >> 172.21.15.28:0/1193772932 conn(0x7f476c06eb10 msgr2=0x7f476c070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 -- 172.21.15.28:0/1193772932 shutdown_connections 2024-09-19T09:39:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.859+0000 7f476bfff640 1 -- 172.21.15.28:0/1193772932 wait complete. 2024-09-19T09:39:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.860+0000 7f476bfff640 1 Processor -- start 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 -- start start 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 0x7f476c1158d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 0x7f476c115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 0x7f476c1b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f476c11c4d0 con 0x7f476c10b4e0 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f476c11c350 con 0x7f476c072d80 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.861+0000 7f476bfff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f476c11c650 con 0x7f476c10de90 2024-09-19T09:39:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 0x7f476c1158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 0x7f476c1b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 0x7f476c1158d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38564/0 (socket says 172.21.15.28:38564) 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476affd640 1 -- 172.21.15.28:0/4042556201 learned_addr learned my addr 172.21.15.28:0/4042556201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476a7fc640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 0x7f476c115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 -- 172.21.15.28:0/4042556201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 msgr2=0x7f476c1158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 0x7f476c1158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 -- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 msgr2=0x7f476c115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 0x7f476c115e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 -- 172.21.15.28:0/4042556201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1b7120 con 0x7f476c10de90 2024-09-19T09:39:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476a7fc640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 0x7f476c115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:39:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.862+0000 7f476b7fe640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 0x7f476c1b6be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476000b3e0 tx=0x7f476000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.871+0000 7f47709a4640 1 -- 172.21.15.28:0/4042556201 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760015020 con 0x7f476c10de90 2024-09-19T09:39:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.871+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1b72f0 con 0x7f476c10de90 2024-09-19T09:39:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.871+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1b7830 con 0x7f476c10de90 2024-09-19T09:39:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.871+0000 7f47709a4640 1 -- 172.21.15.28:0/4042556201 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760011d10 con 0x7f476c10de90 2024-09-19T09:39:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.873+0000 7f47709a4640 1 -- 172.21.15.28:0/4042556201 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476000f2e0 con 0x7f476c10de90 2024-09-19T09:39:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.876+0000 7f47709a4640 1 -- 172.21.15.28:0/4042556201 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4760018020 con 0x7f476c10de90 2024-09-19T09:39:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.877+0000 7f47709a4640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 0x7f47540a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.877+0000 7f47709a4640 1 -- 172.21.15.28:0/4042556201 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f47600cf020 con 0x7f476c10de90 2024-09-19T09:39:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.879+0000 7f476affd640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 0x7f47540a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.880+0000 7f476affd640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 0x7f47540a5800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f475c0095f0 tx=0x7f475c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.881+0000 7f477481f740 1 Processor -- start 2024-09-19T09:39:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.882+0000 7f477481f740 1 -- start start 2024-09-19T09:39:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f477481f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f7cb70 0x55ee96f7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f477481f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50196/0 (socket says 172.21.15.28:50196) 2024-09-19T09:39:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f476a7fc640 1 -- 172.21.15.28:0/3109565614 learned_addr learned my addr 172.21.15.28:0/3109565614 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.883+0000 7f476affd640 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f7cb70 0x55ee96f7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f477481f740 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f84a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee96ee02f0 con 0x55ee96f7d4b0 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee96e88270 con 0x55ee96f82540 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee96edc2b0 con 0x55ee96f7cb70 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f476b7fe640 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f476a7fc640 1 -- 172.21.15.28:0/3109565614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f7cb70 msgr2=0x55ee96f7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f476a7fc640 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f7cb70 0x55ee96f7cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.884+0000 7f476a7fc640 1 -- 172.21.15.28:0/3109565614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 msgr2=0x55ee96f84a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.885+0000 7f476a7fc640 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f84a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.885+0000 7f476a7fc640 1 -- 172.21.15.28:0/3109565614 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee96e87340 con 0x55ee96f7d4b0 2024-09-19T09:39:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.885+0000 7f476a7fc640 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f476402af20 tx=0x7f47640097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327f9cdf662a6e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.946+0000 7f4751ffb640 1 -- 172.21.15.28:0/3109565614 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764031480 con 0x55ee96f7d4b0 2024-09-19T09:39:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.946+0000 7f4751ffb640 1 -- 172.21.15.28:0/3109565614 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4764036040 con 0x55ee96f7d4b0 2024-09-19T09:39:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.948+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 msgr2=0x55ee96f81f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.948+0000 7f477481f740 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f476402af20 tx=0x7f47640097f0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 shutdown_connections 2024-09-19T09:39:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f84a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7d4b0 0x55ee96f81f70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 --2- 172.21.15.28:0/3109565614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f7cb70 0x55ee96f7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 >> 172.21.15.28:0/3109565614 conn(0x55ee96f781e0 msgr2=0x55ee96f785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 shutdown_connections 2024-09-19T09:39:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 -- 172.21.15.28:0/3109565614 wait complete. 2024-09-19T09:39:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.949+0000 7f477481f740 1 -- start start 2024-09-19T09:39:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.950+0000 7f477481f740 1 Processor -- start 2024-09-19T09:39:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96f87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f8e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee96f854a0 con 0x55ee96f82540 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee96e69290 con 0x55ee96f7cb70 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee96f85620 con 0x55ee96f8c250 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.951+0000 7f477481f740 10 client.0 _ll_register_callbacks cb 0x55ee96e9a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38576/0 (socket says 172.21.15.28:38576) 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 -- 172.21.15.28:0/2976949981 learned_addr learned my addr 172.21.15.28:0/2976949981 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476a7fc640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96f87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476b7fe640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f8e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 -- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 msgr2=0x55ee96f8e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f8e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.952+0000 7f476affd640 1 -- 172.21.15.28:0/2976949981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 msgr2=0x55ee96f87290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.953+0000 7f476affd640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96f87290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.953+0000 7f476affd640 1 -- 172.21.15.28:0/2976949981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee96bf3210 con 0x55ee96f7cb70 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.953+0000 7f476a7fc640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96f87290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:39:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.953+0000 7f476affd640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f475c0094a0 tx=0x7f475c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7b7a36213c9bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.974+0000 7f474affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:39:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.974+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c01be60 con 0x55ee96f7cb70 2024-09-19T09:39:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.974+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee96eeda20 con 0x55ee96f7cb70 2024-09-19T09:39:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.975+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f475c01ad20 con 0x55ee96f7cb70 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.975+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f475c0263f0 con 0x55ee96f7cb70 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.975+0000 7f477481f740 10 client.26258 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.975+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee96cbf620 con 0x55ee96f7cb70 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.975+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ee96eeda20 con 0x55ee96f7cb70 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f475c025070 con 0x55ee96f7cb70 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 10 client.26258 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 10 client.26258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 20 client.26258 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96f93340 0x55ee96f957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 4 client.26258 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:39:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee96ee7d70 con 0x55ee96f93340 2024-09-19T09:39:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.976+0000 7f476a7fc640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96f93340 0x55ee96f957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.979+0000 7f476a7fc640 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96f93340 0x55ee96f957a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86367c942d8213fe server_cookie=cb520fc443cd3bb3 in_seq=0 out_seq=0 2024-09-19T09:39:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.979+0000 7f474affd640 10 client.26258 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.985+0000 7f474affd640 1 -- 172.21.15.28:0/2976949981 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee96ee7d70 con 0x55ee96f93340 2024-09-19T09:39:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.985+0000 7f474affd640 10 client.26258 handle_command_reply: tid=0 2024-09-19T09:39:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.985+0000 7f477481f740 1 client.26258 shutdown 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 msgr2=0x55ee96f86d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f475c0094a0 tx=0x7f475c013040 comp rx=0 tx=0).stop 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 shutdown_connections 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96f93340 0x55ee96f957a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f8e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96f87290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 --2- 172.21.15.28:0/2976949981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96f86d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 >> 172.21.15.28:0/2976949981 conn(0x55ee96f781e0 msgr2=0x55ee96f79240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.986+0000 7f474affd640 0 client.26258 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.987+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 shutdown_connections 2024-09-19T09:39:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.988+0000 7f477481f740 1 -- 172.21.15.28:0/2976949981 wait complete. 2024-09-19T09:39:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:08.988+0000 7f477481f740 20 client.26258 trim_cache size 0 max 16384 2024-09-19T09:39:09.059 INFO:tasks.workunit.client.0.smithi028.stdout: 169 1984659 627 1447184 403 1049352 1558 2024-09-19T09:39:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.098+0000 7f477481f740 1 Processor -- start 2024-09-19T09:39:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.098+0000 7f477481f740 1 -- start start 2024-09-19T09:39:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.099+0000 7f477481f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7cb70 0x55ee96f7e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f477481f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f477481f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f477481f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f475800e4c0 con 0x55ee96f7cb70 2024-09-19T09:39:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f476affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7cb70 0x55ee96f7e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38588/0 (socket says 172.21.15.28:38588) 2024-09-19T09:39:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f476a7fc640 1 -- 172.21.15.28:0/39451670 learned_addr learned my addr 172.21.15.28:0/39451670 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.110+0000 7f476b7fe640 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4758010030 con 0x55ee96f82540 2024-09-19T09:39:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f475800eb90 con 0x55ee96f8c250 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f476a7fc640 1 -- 172.21.15.28:0/39451670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 msgr2=0x55ee96f86b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f476a7fc640 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f86b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f476a7fc640 1 -- 172.21.15.28:0/39451670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7cb70 msgr2=0x55ee96f7e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f476a7fc640 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7cb70 0x55ee96f7e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.111+0000 7f476a7fc640 1 -- 172.21.15.28:0/39451670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee96f86890 con 0x55ee96f82540 2024-09-19T09:39:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.112+0000 7f476a7fc640 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f476402af20 tx=0x7f4764009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35679b417490f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.125+0000 7f47517fa640 1 -- 172.21.15.28:0/39451670 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764031480 con 0x55ee96f82540 2024-09-19T09:39:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.125+0000 7f47517fa640 1 -- 172.21.15.28:0/39451670 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4764036040 con 0x55ee96f82540 2024-09-19T09:39:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.125+0000 7f47517fa640 1 -- 172.21.15.28:0/39451670 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476404ea20 con 0x55ee96f82540 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 msgr2=0x55ee96f7ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f476402af20 tx=0x7f4764009780 comp rx=0 tx=0).stop 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 shutdown_connections 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f82540 0x55ee96f7ea90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96f86b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 --2- 172.21.15.28:0/39451670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f7cb70 0x55ee96f7e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.129+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 >> 172.21.15.28:0/39451670 conn(0x55ee96f781e0 msgr2=0x55ee96f785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.130+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 shutdown_connections 2024-09-19T09:39:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.130+0000 7f477481f740 1 -- 172.21.15.28:0/39451670 wait complete. 2024-09-19T09:39:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.131+0000 7f477481f740 1 -- start start 2024-09-19T09:39:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.131+0000 7f477481f740 1 Processor -- start 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.131+0000 7f477481f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96fa2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96ffbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f475800f960 con 0x55ee96f82540 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f475800f260 con 0x55ee96f7cb70 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f475800c4a0 con 0x55ee96f8c250 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f477481f740 10 client.0 _ll_register_callbacks cb 0x7f475c0097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f476affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:38594/0 (socket says 172.21.15.28:38594) 2024-09-19T09:39:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f476affd640 1 -- 172.21.15.28:0/570004245 learned_addr learned my addr 172.21.15.28:0/570004245 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.132+0000 7f476b7fe640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96ffbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.133+0000 7f476affd640 1 -- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 msgr2=0x55ee96ffbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.133+0000 7f476affd640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96ffbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.133+0000 7f476affd640 1 -- 172.21.15.28:0/570004245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 msgr2=0x55ee96fa2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.133+0000 7f476affd640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96fa2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.133+0000 7f476affd640 1 -- 172.21.15.28:0/570004245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee96ffc4c0 con 0x55ee96f7cb70 2024-09-19T09:39:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.136+0000 7f476affd640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f475c01a760 tx=0x7f475c0326b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de68203d92b6a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.141+0000 7f474affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:39:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.141+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c01be60 con 0x55ee96f7cb70 2024-09-19T09:39:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.141+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4764036040 con 0x55ee96f7cb70 2024-09-19T09:39:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.141+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f475c005040 con 0x55ee96f7cb70 2024-09-19T09:39:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.142+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f475c045050 con 0x55ee96f7cb70 2024-09-19T09:39:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.144+0000 7f477481f740 10 client.26264 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.144+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee96ffd860 con 0x55ee96f7cb70 2024-09-19T09:39:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.144+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4764036040 con 0x55ee96f7cb70 2024-09-19T09:39:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.144+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f475c03c070 con 0x55ee96f7cb70 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 10 client.26264 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 20 client.26264 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96ffe1d0 0x55ee970005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 4 client.26264 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee96e64900 con 0x55ee96ffe1d0 2024-09-19T09:39:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f476a7fc640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96ffe1d0 0x55ee970005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.145+0000 7f476a7fc640 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96ffe1d0 0x55ee970005f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281f000ff5fee114 server_cookie=bd7abb149338a29f in_seq=0 out_seq=0 2024-09-19T09:39:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:09.146+0000 7f474affd640 10 client.26264 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:09.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:09 smithi078 ceph-mon[35619]: pgmap v533: 97 pgs: 97 active+clean; 538 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-19T09:39:09.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:09 smithi038 ceph-mon[35271]: pgmap v533: 97 pgs: 97 active+clean; 538 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-19T09:39:09.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:09 smithi028 ceph-mon[30972]: pgmap v533: 97 pgs: 97 active+clean; 538 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-19T09:39:11.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:11 smithi078 ceph-mon[35619]: pgmap v534: 97 pgs: 97 active+clean; 541 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 397 op/s 2024-09-19T09:39:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:11 smithi038 ceph-mon[35271]: pgmap v534: 97 pgs: 97 active+clean; 541 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 397 op/s 2024-09-19T09:39:11.767 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:11 smithi028 ceph-mon[30972]: pgmap v534: 97 pgs: 97 active+clean; 541 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 397 op/s 2024-09-19T09:39:13.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:13 smithi078 ceph-mon[35619]: pgmap v535: 97 pgs: 97 active+clean; 540 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-09-19T09:39:13.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:13 smithi038 ceph-mon[35271]: pgmap v535: 97 pgs: 97 active+clean; 540 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-09-19T09:39:13.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:13 smithi028 ceph-mon[30972]: pgmap v535: 97 pgs: 97 active+clean; 540 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.8 MiB/s wr, 328 op/s 2024-09-19T09:39:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.260+0000 7f474affd640 1 -- 172.21.15.28:0/570004245 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee96e64900 con 0x55ee96ffe1d0 2024-09-19T09:39:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.260+0000 7f474affd640 10 client.26264 handle_command_reply: tid=0 2024-09-19T09:39:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.264+0000 7f477481f740 1 client.26264 shutdown 2024-09-19T09:39:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 msgr2=0x55ee96fa2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f475c01a760 tx=0x7f475c0326b0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 shutdown_connections 2024-09-19T09:39:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ee96ffe1d0 0x55ee970005f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96f8c250 0x55ee96ffbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ee96f82540 0x55ee96fa2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 --2- 172.21.15.28:0/570004245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee96f7cb70 0x55ee96fa2740 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.265+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 >> 172.21.15.28:0/570004245 conn(0x55ee96f781e0 msgr2=0x55ee96f86b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.266+0000 7f474affd640 0 client.26264 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.266+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 shutdown_connections 2024-09-19T09:39:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.266+0000 7f477481f740 1 -- 172.21.15.28:0/570004245 wait complete. 2024-09-19T09:39:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.266+0000 7f477481f740 20 client.26264 trim_cache size 0 max 16384 2024-09-19T09:39:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:39:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:39:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:39:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:39:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 msgr2=0x7f47540a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 0x7f47540a5800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f475c0095f0 tx=0x7f475c005f90 comp rx=0 tx=0).stop 2024-09-19T09:39:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 msgr2=0x7f476c1b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 0x7f476c1b6be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476000b3e0 tx=0x7f476000b8a0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 shutdown_connections 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f47540a3340 0x7f47540a5800 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f476c10de90 0x7f476c1b6be0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f476c10b4e0 0x7f476c115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.283+0000 7f476bfff640 1 --2- 172.21.15.28:0/4042556201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c072d80 0x7f476c1158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.284+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 >> 172.21.15.28:0/4042556201 conn(0x7f476c06eb10 msgr2=0x7f476c10f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.285+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 shutdown_connections 2024-09-19T09:39:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:14.286+0000 7f476bfff640 1 -- 172.21.15.28:0/4042556201 wait complete. 2024-09-19T09:39:15.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:15 smithi078 ceph-mon[35619]: pgmap v536: 97 pgs: 97 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.9 MiB/s wr, 797 op/s 2024-09-19T09:39:15.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:15 smithi038 ceph-mon[35271]: pgmap v536: 97 pgs: 97 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.9 MiB/s wr, 797 op/s 2024-09-19T09:39:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:15 smithi028 ceph-mon[30972]: pgmap v536: 97 pgs: 97 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.9 MiB/s wr, 797 op/s 2024-09-19T09:39:17.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:17 smithi078 ceph-mon[35619]: pgmap v537: 97 pgs: 97 active+clean; 530 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-09-19T09:39:17.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:17 smithi038 ceph-mon[35271]: pgmap v537: 97 pgs: 97 active+clean; 530 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-09-19T09:39:17.777 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:17 smithi028 ceph-mon[30972]: pgmap v537: 97 pgs: 97 active+clean; 530 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 1.00k op/s 2024-09-19T09:39:19.059 INFO:tasks.workunit.client.0.smithi028.stdout: 176 1762762 417 1272396 314 945739 1209 2024-09-19T09:39:19.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:19 smithi078 ceph-mon[35619]: pgmap v538: 97 pgs: 97 active+clean; 524 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 1.07k op/s 2024-09-19T09:39:19.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:19 smithi038 ceph-mon[35271]: pgmap v538: 97 pgs: 97 active+clean; 524 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 1.07k op/s 2024-09-19T09:39:19.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:19 smithi028 ceph-mon[30972]: pgmap v538: 97 pgs: 97 active+clean; 524 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 1.07k op/s 2024-09-19T09:39:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:21 smithi078 ceph-mon[35619]: pgmap v539: 97 pgs: 97 active+clean; 539 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-09-19T09:39:21.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:21 smithi038 ceph-mon[35271]: pgmap v539: 97 pgs: 97 active+clean; 539 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-09-19T09:39:21.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:21 smithi028 ceph-mon[30972]: pgmap v539: 97 pgs: 97 active+clean; 539 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-09-19T09:39:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:23 smithi078 ceph-mon[35619]: pgmap v540: 97 pgs: 97 active+clean; 554 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 1.30k op/s 2024-09-19T09:39:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:23 smithi038 ceph-mon[35271]: pgmap v540: 97 pgs: 97 active+clean; 554 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 1.30k op/s 2024-09-19T09:39:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:23 smithi028 ceph-mon[30972]: pgmap v540: 97 pgs: 97 active+clean; 554 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 1.30k op/s 2024-09-19T09:39:25.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:25 smithi078 ceph-mon[35619]: pgmap v541: 97 pgs: 97 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.1 MiB/s wr, 1.45k op/s 2024-09-19T09:39:25.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:25 smithi038 ceph-mon[35271]: pgmap v541: 97 pgs: 97 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.1 MiB/s wr, 1.45k op/s 2024-09-19T09:39:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:25 smithi028 ceph-mon[30972]: pgmap v541: 97 pgs: 97 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.1 MiB/s wr, 1.45k op/s 2024-09-19T09:39:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:27 smithi078 ceph-mon[35619]: pgmap v542: 97 pgs: 97 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.01k op/s 2024-09-19T09:39:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:27 smithi038 ceph-mon[35271]: pgmap v542: 97 pgs: 97 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.01k op/s 2024-09-19T09:39:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:27 smithi028 ceph-mon[30972]: pgmap v542: 97 pgs: 97 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.01k op/s 2024-09-19T09:39:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.979+0000 7f1b64fcf640 1 -- 172.21.15.28:0/1585008753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 msgr2=0x7f1b60119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.979+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/1585008753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b60119130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009ca0 tx=0x7f1b5802a8f0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.980+0000 7f1b64fcf640 1 -- 172.21.15.28:0/1585008753 shutdown_connections 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.980+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/1585008753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b60119130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.980+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/1585008753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b601091f0 0x7f1b60109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.980+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/1585008753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b600732d0 0x7f1b600736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.980+0000 7f1b64fcf640 1 -- 172.21.15.28:0/1585008753 >> 172.21.15.28:0/1585008753 conn(0x7f1b6006eb10 msgr2=0x7f1b60070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.981+0000 7f1b64fcf640 1 -- 172.21.15.28:0/1585008753 shutdown_connections 2024-09-19T09:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.981+0000 7f1b64fcf640 1 -- 172.21.15.28:0/1585008753 wait complete. 2024-09-19T09:39:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 Processor -- start 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 -- start start 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b601091f0 0x7f1b60115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b601b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6011c520 con 0x7f1b600732d0 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6011c3a0 con 0x7f1b601091f0 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.982+0000 7f1b64fcf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6011c6a0 con 0x7f1b60109bb0 2024-09-19T09:39:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.985+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.985+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58700/0 (socket says 172.21.15.28:58700) 2024-09-19T09:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.985+0000 7f1b5e59c640 1 -- 172.21.15.28:0/324647915 learned_addr learned my addr 172.21.15.28:0/324647915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.985+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b601091f0 0x7f1b60115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.985+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b601b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 -- 172.21.15.28:0/324647915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 msgr2=0x7f1b601b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b601b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 -- 172.21.15.28:0/324647915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b601091f0 msgr2=0x7f1b60115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b601091f0 0x7f1b60115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 -- 172.21.15.28:0/324647915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b60118e70 con 0x7f1b600732d0 2024-09-19T09:39:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:28.986+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9c0 tx=0x7f1b5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.040+0000 7f1b4f7fe640 1 -- 172.21.15.28:0/324647915 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000cd80 con 0x7f1b600732d0 2024-09-19T09:39:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.041+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b601b71d0 con 0x7f1b600732d0 2024-09-19T09:39:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.041+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601b76f0 con 0x7f1b600732d0 2024-09-19T09:39:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.044+0000 7f1b4f7fe640 1 -- 172.21.15.28:0/324647915 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b500035f0 con 0x7f1b600732d0 2024-09-19T09:39:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.044+0000 7f1b4f7fe640 1 -- 172.21.15.28:0/324647915 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50010690 con 0x7f1b600732d0 2024-09-19T09:39:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 Processor -- start 2024-09-19T09:39:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 -- start start 2024-09-19T09:39:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0233fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c023ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5649c01972f0 con 0x5649c0239570 2024-09-19T09:39:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5649c013f270 con 0x5649c02344e0 2024-09-19T09:39:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.050+0000 7f1b67e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649c01932b0 con 0x5649c0233ba0 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:59046/0 (socket says 172.21.15.28:59046) 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 -- 172.21.15.28:0/2678043544 learned_addr learned my addr 172.21.15.28:0/2678043544 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c023ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0233fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 -- 172.21.15.28:0/2678043544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 msgr2=0x5649c0233fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0233fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 -- 172.21.15.28:0/2678043544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 msgr2=0x5649c023ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c023ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5e59c640 1 -- 172.21.15.28:0/2678043544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c013e340 con 0x5649c02344e0 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.051+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c023ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.052+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b50003950 tx=0x7f1b50004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca73384e7b21d258 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.052+0000 7f1b4f7fe640 1 -- 172.21.15.28:0/324647915 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1b500108f0 con 0x7f1b600732d0 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.052+0000 7f1b4cff9640 1 -- 172.21.15.28:0/2678043544 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b500d6420 con 0x5649c02344e0 2024-09-19T09:39:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.052+0000 7f1b4cff9640 1 -- 172.21.15.28:0/2678043544 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b500e0070 con 0x5649c02344e0 2024-09-19T09:39:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b4f7fe640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 0x7f1b480a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 -- 172.21.15.28:0/2678043544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 msgr2=0x5649c0238fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b50003950 tx=0x7f1b50004730 comp rx=0 tx=0).stop 2024-09-19T09:39:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 0x7f1b480a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 -- 172.21.15.28:0/2678043544 shutdown_connections 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0238fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c023ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 --2- 172.21.15.28:0/2678043544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0233fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.054+0000 7f1b67e45740 1 -- 172.21.15.28:0/2678043544 >> 172.21.15.28:0/2678043544 conn(0x5649c022f210 msgr2=0x5649c022f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.055+0000 7f1b67e45740 1 -- 172.21.15.28:0/2678043544 shutdown_connections 2024-09-19T09:39:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.055+0000 7f1b67e45740 1 -- 172.21.15.28:0/2678043544 wait complete. 2024-09-19T09:39:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.056+0000 7f1b67e45740 1 -- start start 2024-09-19T09:39:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.056+0000 7f1b67e45740 1 Processor -- start 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0233ba0 0x5649c023e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0245650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5649c023c590 con 0x5649c0239570 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5649c0120290 con 0x5649c0233ba0 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649c023c710 con 0x5649c02344e0 2024-09-19T09:39:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b67e45740 10 client.0 _ll_register_callbacks cb 0x5649c0151140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:29.059 INFO:tasks.workunit.client.0.smithi028.stdout: 180 1619894 349 1179425 332 876989 1362 2024-09-19T09:39:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36794/0 (socket says 172.21.15.28:36794) 2024-09-19T09:39:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b5dd9b640 1 -- 172.21.15.28:0/3017725624 learned_addr learned my addr 172.21.15.28:0/3017725624 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0233ba0 0x5649c023e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.057+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0245650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 -- 172.21.15.28:0/3017725624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0233ba0 msgr2=0x5649c023e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0233ba0 0x5649c023e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 -- 172.21.15.28:0/3017725624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 msgr2=0x5649c0245650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0245650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 -- 172.21.15.28:0/3017725624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649bfeaa210 con 0x5649c02344e0 2024-09-19T09:39:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.058+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009f20 tx=0x7f1b540116f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50722d80ae4b41c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.062+0000 7f1b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.063+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54019070 con 0x5649c02344e0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.063+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649c01a4a20 con 0x5649c02344e0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.063+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1b54015870 con 0x5649c02344e0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.063+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b54025050 con 0x5649c02344e0 2024-09-19T09:39:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.063+0000 7f1b67e45740 10 client.26263 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.064+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5649bff76620 con 0x5649c02344e0 2024-09-19T09:39:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.064+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5649c01a4a20 con 0x5649c02344e0 2024-09-19T09:39:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.064+0000 7f1b4f7fe640 1 -- 172.21.15.28:0/324647915 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1b500d0e70 con 0x7f1b600732d0 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 0x7f1b480a5920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006010 tx=0x7f1b58004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f1b54014be0 con 0x5649c02344e0 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 10 client.26263 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 20 client.26263 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c024a2a0 0x5649c024c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 4 client.26263 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:39:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.066+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c019ed70 con 0x5649c024a2a0 2024-09-19T09:39:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.067+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c024a2a0 0x5649c024c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.068+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c024a2a0 0x5649c024c720 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684427fa842a711b server_cookie=8ccdadb4f35a314b in_seq=0 out_seq=0 2024-09-19T09:39:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.068+0000 7f1b45ffb640 10 client.26263 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.070+0000 7f1b45ffb640 1 -- 172.21.15.28:0/3017725624 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649c019ed70 con 0x5649c024a2a0 2024-09-19T09:39:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.070+0000 7f1b45ffb640 10 client.26263 handle_command_reply: tid=0 2024-09-19T09:39:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.070+0000 7f1b67e45740 1 client.26263 shutdown 2024-09-19T09:39:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 msgr2=0x5649c023e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009f20 tx=0x7f1b540116f0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 shutdown_connections 2024-09-19T09:39:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c024a2a0 0x5649c024c720 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0245650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c02344e0 0x5649c023e610 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 --2- 172.21.15.28:0/3017725624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0233ba0 0x5649c023e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 >> 172.21.15.28:0/3017725624 conn(0x5649c022f210 msgr2=0x5649c0230a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b45ffb640 0 client.26263 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 shutdown_connections 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.071+0000 7f1b67e45740 1 -- 172.21.15.28:0/3017725624 wait complete. 2024-09-19T09:39:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.072+0000 7f1b67e45740 20 client.26263 trim_cache size 0 max 16384 2024-09-19T09:39:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.159+0000 7f1b67e45740 1 Processor -- start 2024-09-19T09:39:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.160+0000 7f1b67e45740 1 -- start start 2024-09-19T09:39:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.160+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.160+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36802/0 (socket says 172.21.15.28:36802) 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58720/0 (socket says 172.21.15.28:58720) 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b5e59c640 1 -- 172.21.15.28:0/4096361685 learned_addr learned my addr 172.21.15.28:0/4096361685 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b67e45740 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0239570 0x5649c023e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000e330 con 0x5649c02344e0 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.161+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000fea0 con 0x5649c0239570 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000ea00 con 0x5649c0233ba0 2024-09-19T09:39:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0239570 0x5649c023e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 -- 172.21.15.28:0/4096361685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0239570 msgr2=0x5649c023e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0239570 0x5649c023e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 -- 172.21.15.28:0/4096361685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 msgr2=0x5649c0245610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 -- 172.21.15.28:0/4096361685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c023dc10 con 0x5649c0233ba0 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.162+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c6a0 tx=0x7f1b500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2320eef681cdde16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.185+0000 7f1b47fff640 1 -- 172.21.15.28:0/4096361685 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5009d230 con 0x5649c0233ba0 2024-09-19T09:39:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.186+0000 7f1b47fff640 1 -- 172.21.15.28:0/4096361685 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b500e0040 con 0x5649c0233ba0 2024-09-19T09:39:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.186+0000 7f1b47fff640 1 -- 172.21.15.28:0/4096361685 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5009cdc0 con 0x5649c0233ba0 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.187+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 msgr2=0x5649c023cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c6a0 tx=0x7f1b500046c0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 shutdown_connections 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c0239570 0x5649c023e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c02344e0 0x5649c0245610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 --2- 172.21.15.28:0/4096361685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c023cb90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.188+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 >> 172.21.15.28:0/4096361685 conn(0x5649c022f210 msgr2=0x5649c022f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.197+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 shutdown_connections 2024-09-19T09:39:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.197+0000 7f1b67e45740 1 -- 172.21.15.28:0/4096361685 wait complete. 2024-09-19T09:39:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.198+0000 7f1b67e45740 1 -- start start 2024-09-19T09:39:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.198+0000 7f1b67e45740 1 Processor -- start 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0245520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b67e45740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:36814/0 (socket says 172.21.15.28:36814) 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0245520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.201+0000 7f1b5dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0245520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:59070/0 (socket says 172.21.15.28:59070) 2024-09-19T09:39:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58732/0 (socket says 172.21.15.28:58732) 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 -- 172.21.15.28:0/1620384523 learned_addr learned my addr 172.21.15.28:0/1620384523 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000f7d0 con 0x5649c0239570 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000f0d0 con 0x5649c02344e0 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4000c3c0 con 0x5649c0233ba0 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 -- 172.21.15.28:0/1620384523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 msgr2=0x5649c0245520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0245520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 -- 172.21.15.28:0/1620384523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 msgr2=0x5649c0241b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5e59c640 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c0242340 con 0x5649c0233ba0 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.202+0000 7f1b5ed9d640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.203+0000 7f1b5e59c640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1b5001f000 tx=0x7f1b500037a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b78a3100c8ce37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.203+0000 7f1b67e45740 10 client.0 _ll_register_callbacks cb 0x7f1b54009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.203+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b500e0040 con 0x5649c0233ba0 2024-09-19T09:39:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.222+0000 7f1b45ffb640 10 client.26269 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:39:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.222+0000 7f1b45ffb640 1 -- 172.21.15.28:0/1620384523 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5009cfa0 con 0x5649c0233ba0 2024-09-19T09:39:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.222+0000 7f1b45ffb640 1 -- 172.21.15.28:0/1620384523 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f1b500d4380 con 0x5649c0233ba0 2024-09-19T09:39:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.223+0000 7f1b45ffb640 1 -- 172.21.15.28:0/1620384523 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b5009cdc0 con 0x5649c0233ba0 2024-09-19T09:39:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.223+0000 7f1b67e45740 10 client.26269 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.223+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5649c02b3a40 con 0x5649c0233ba0 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b45ffb640 1 -- 172.21.15.28:0/1620384523 <== mon.1 v2:172.21.15.78:3300/0 4 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f1b50099ad0 con 0x5649c0233ba0 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 10 client.26269 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 20 client.26269 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c02b4180 0x5649c02b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 4 client.26269 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:39:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1b500e0040 con 0x5649c02b4180 2024-09-19T09:39:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.224+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c02b4180 0x5649c02b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.225+0000 7f1b5dd9b640 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c02b4180 0x5649c02b6640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c1ebaf9baa3c5b server_cookie=94f5901037b67a9c in_seq=0 out_seq=0 2024-09-19T09:39:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:29.227+0000 7f1b45ffb640 10 client.26269 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:29 smithi078 ceph-mon[35619]: pgmap v543: 97 pgs: 97 active+clean; 591 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T09:39:29.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:29 smithi038 ceph-mon[35271]: pgmap v543: 97 pgs: 97 active+clean; 591 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T09:39:29.766 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:29 smithi028 ceph-mon[30972]: pgmap v543: 97 pgs: 97 active+clean; 591 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T09:39:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:31 smithi038 ceph-mon[35271]: pgmap v544: 97 pgs: 97 active+clean; 590 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 643 op/s 2024-09-19T09:39:31.767 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:31 smithi028 ceph-mon[30972]: pgmap v544: 97 pgs: 97 active+clean; 590 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 643 op/s 2024-09-19T09:39:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:31 smithi078 ceph-mon[35619]: pgmap v544: 97 pgs: 97 active+clean; 590 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 643 op/s 2024-09-19T09:39:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:33 smithi038 ceph-mon[35271]: pgmap v545: 97 pgs: 97 active+clean; 593 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-09-19T09:39:33.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:33 smithi028 ceph-mon[30972]: pgmap v545: 97 pgs: 97 active+clean; 593 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-09-19T09:39:33.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:33 smithi078 ceph-mon[35619]: pgmap v545: 97 pgs: 97 active+clean; 593 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-09-19T09:39:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b45ffb640 1 -- 172.21.15.28:0/1620384523 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1b500e0040 con 0x5649c02b4180 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b45ffb640 10 client.26269 handle_command_reply: tid=0 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 client.26269 shutdown 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 msgr2=0x5649c0244fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1b5001f000 tx=0x7f1b500037a0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 shutdown_connections 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649c02344e0 0x5649c0245520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5649c02b4180 0x5649c02b6640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5649c0239570 0x5649c0241b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 --2- 172.21.15.28:0/1620384523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649c0233ba0 0x5649c0244fe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 >> 172.21.15.28:0/1620384523 conn(0x5649c022f210 msgr2=0x5649c023e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b45ffb640 0 client.26269 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 shutdown_connections 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 1 -- 172.21.15.28:0/1620384523 wait complete. 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.772+0000 7f1b67e45740 20 client.26269 trim_cache size 0 max 16384 2024-09-19T09:39:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:39:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:39:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:39:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:39:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 msgr2=0x7f1b480a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 0x7f1b480a5920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006010 tx=0x7f1b58004a60 comp rx=0 tx=0).stop 2024-09-19T09:39:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 msgr2=0x7f1b601157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9c0 tx=0x7f1b5000ee80 comp rx=0 tx=0).stop 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 shutdown_connections 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f1b480a3460 0x7f1b480a5920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b601091f0 0x7f1b60115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b60109bb0 0x7f1b601b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 --2- 172.21.15.28:0/324647915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b600732d0 0x7f1b601157b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 >> 172.21.15.28:0/324647915 conn(0x7f1b6006eb10 msgr2=0x7f1b60071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 shutdown_connections 2024-09-19T09:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:34.788+0000 7f1b64fcf640 1 -- 172.21.15.28:0/324647915 wait complete. 2024-09-19T09:39:35.659 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:35 smithi078 ceph-mon[35619]: pgmap v546: 97 pgs: 97 active+clean; 598 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 771 op/s 2024-09-19T09:39:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:35 smithi038 ceph-mon[35271]: pgmap v546: 97 pgs: 97 active+clean; 598 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 771 op/s 2024-09-19T09:39:35.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:35 smithi028 ceph-mon[30972]: pgmap v546: 97 pgs: 97 active+clean; 598 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 771 op/s 2024-09-19T09:39:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:37 smithi038 ceph-mon[35271]: pgmap v547: 97 pgs: 97 active+clean; 597 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 978 op/s 2024-09-19T09:39:37.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:37 smithi028 ceph-mon[30972]: pgmap v547: 97 pgs: 97 active+clean; 597 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 978 op/s 2024-09-19T09:39:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:37 smithi078 ceph-mon[35619]: pgmap v547: 97 pgs: 97 active+clean; 597 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 978 op/s 2024-09-19T09:39:39.059 INFO:tasks.workunit.client.0.smithi028.stdout: 186 1906424 534 1382798 359 1068068 1637 2024-09-19T09:39:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:39 smithi038 ceph-mon[35271]: pgmap v548: 97 pgs: 97 active+clean; 576 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.02k op/s 2024-09-19T09:39:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:39 smithi028 ceph-mon[30972]: pgmap v548: 97 pgs: 97 active+clean; 576 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.02k op/s 2024-09-19T09:39:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:39 smithi078 ceph-mon[35619]: pgmap v548: 97 pgs: 97 active+clean; 576 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 1.02k op/s 2024-09-19T09:39:40.498 INFO:tasks.workunit.client.0.smithi028.stdout: 2024-09-19T09:39:40.499 INFO:tasks.workunit.client.0.smithi028.stdout:Final score for writes: 186 2024-09-19T09:39:40.499 INFO:tasks.workunit.client.0.smithi028.stdout:Final score for reads : 560474 2024-09-19T09:39:40.499 INFO:tasks.workunit.client.0.smithi028.stdout: 2024-09-19T09:39:40.501 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:39:40.501 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:39:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:41 smithi038 ceph-mon[35271]: pgmap v549: 97 pgs: 97 active+clean; 560 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-09-19T09:39:41.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:41 smithi028 ceph-mon[30972]: pgmap v549: 97 pgs: 97 active+clean; 560 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-09-19T09:39:41.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:41 smithi078 ceph-mon[35619]: pgmap v549: 97 pgs: 97 active+clean; 560 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-09-19T09:39:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:43 smithi038 ceph-mon[35271]: pgmap v550: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-19T09:39:43.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:43 smithi028 ceph-mon[30972]: pgmap v550: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-19T09:39:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:43 smithi078 ceph-mon[35619]: pgmap v550: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-19T09:39:45.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:45 smithi038 ceph-mon[35271]: pgmap v551: 97 pgs: 97 active+clean; 578 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.09k op/s 2024-09-19T09:39:45.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:45 smithi028 ceph-mon[30972]: pgmap v551: 97 pgs: 97 active+clean; 578 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.09k op/s 2024-09-19T09:39:45.838 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:39:45.838 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T09:39:45.839 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:39:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:45 smithi078 ceph-mon[35619]: pgmap v551: 97 pgs: 97 active+clean; 578 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.09k op/s 2024-09-19T09:39:46.129 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:39:46.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 -- 172.21.15.28:0/2085182903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a8106f20 msgr2=0x7fa3a810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:46.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 --2- 172.21.15.28:0/2085182903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a8106f20 0x7fa3a810eaa0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009ca0 tx=0x7fa3a002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 -- 172.21.15.28:0/2085182903 shutdown_connections 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 --2- 172.21.15.28:0/2085182903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a81027a0 0x7fa3a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 --2- 172.21.15.28:0/2085182903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a8106f20 0x7fa3a810eaa0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 --2- 172.21.15.28:0/2085182903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8101df0 0x7fa3a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 -- 172.21.15.28:0/2085182903 >> 172.21.15.28:0/2085182903 conn(0x7fa3a80fd8f0 msgr2=0x7fa3a80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.608+0000 7fa3af845640 1 -- 172.21.15.28:0/2085182903 shutdown_connections 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.609+0000 7fa3af845640 1 -- 172.21.15.28:0/2085182903 wait complete. 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.609+0000 7fa3af845640 1 Processor -- start 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.609+0000 7fa3af845640 1 -- start start 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a8101df0 0x7fa3a81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8106f20 0x7fa3a810dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8111ff0 con 0x7fa3a81027a0 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8111e70 con 0x7fa3a8101df0 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3af845640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8112170 con 0x7fa3a8106f20 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3ad5e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a8101df0 0x7fa3a81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48888/0 (socket says 172.21.15.28:48888) 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 -- 172.21.15.28:0/2126167203 learned_addr learned my addr 172.21.15.28:0/2126167203 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3adde2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8106f20 0x7fa3a810dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 -- 172.21.15.28:0/2126167203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8106f20 msgr2=0x7fa3a810dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8106f20 0x7fa3a810dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 -- 172.21.15.28:0/2126167203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a8101df0 msgr2=0x7fa3a81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:46.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a8101df0 0x7fa3a81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.610+0000 7fa3acde0640 1 -- 172.21.15.28:0/2126167203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a810e1e0 con 0x7fa3a81027a0 2024-09-19T09:39:46.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa3acde0640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa39800b4a0 tx=0x7fa39800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:46.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398014070 con 0x7fa3a81027a0 2024-09-19T09:39:46.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398010af0 con 0x7fa3a81027a0 2024-09-19T09:39:46.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a810e4d0 con 0x7fa3a81027a0 2024-09-19T09:39:46.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398017d10 con 0x7fa3a81027a0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.611+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a81ab570 con 0x7fa3a81027a0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.612+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a81021d0 con 0x7fa3a81027a0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.613+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa398017070 con 0x7fa3a81027a0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.614+0000 7fa39e7fc640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 0x7fa3940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.614+0000 7fa3ad5e1640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 0x7fa3940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.614+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fa3980cef90 con 0x7fa3a81027a0 2024-09-19T09:39:46.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.614+0000 7fa3ad5e1640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 0x7fa3940a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3a810ebf0 tx=0x7fa3a400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:46.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.617+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa39809b690 con 0x7fa3a81027a0 2024-09-19T09:39:46.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.830+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a802d7c0 con 0x7fa3a81027a0 2024-09-19T09:39:46.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.831+0000 7fa39e7fc640 1 -- 172.21.15.28:0/2126167203 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11954 (secure 0 0 0) 0x7fa39809f990 con 0x7fa3a81027a0 2024-09-19T09:39:46.834 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:39:46.834 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-19T09:38:35:591758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:38:35.429459+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:39:46.834 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 msgr2=0x7fa3940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 0x7fa3940a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3a810ebf0 tx=0x7fa3a400b410 comp rx=0 tx=0).stop 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 msgr2=0x7fa3a81a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa39800b4a0 tx=0x7fa39800b960 comp rx=0 tx=0).stop 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 shutdown_connections 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa3940a3240 0x7fa3940a5700 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3a8106f20 0x7fa3a810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3a81027a0 0x7fa3a81a4be0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 --2- 172.21.15.28:0/2126167203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a8101df0 0x7fa3a81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 >> 172.21.15.28:0/2126167203 conn(0x7fa3a80fd8f0 msgr2=0x7fa3a80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 shutdown_connections 2024-09-19T09:39:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:46.834+0000 7fa3af845640 1 -- 172.21.15.28:0/2126167203 wait complete. 2024-09-19T09:39:47.597 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:47 smithi028 ceph-mon[30972]: pgmap v552: 97 pgs: 97 active+clean; 609 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 844 op/s 2024-09-19T09:39:47.597 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:47 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2126167203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:39:47.624 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T09:39:47.624 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T09:39:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:47 smithi038 ceph-mon[35271]: pgmap v552: 97 pgs: 97 active+clean; 609 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 844 op/s 2024-09-19T09:39:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:47 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2126167203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:39:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:47 smithi078 ceph-mon[35619]: pgmap v552: 97 pgs: 97 active+clean; 609 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 844 op/s 2024-09-19T09:39:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:47 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2126167203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:39:47.886 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:39:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 -- 172.21.15.28:0/4045035283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 msgr2=0x7f6a50103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 --2- 172.21.15.28:0/4045035283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50103260 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009900 tx=0x7f6a3802a3f0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 -- 172.21.15.28:0/4045035283 shutdown_connections 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 --2- 172.21.15.28:0/4045035283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a501037a0 0x7f6a5010a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 --2- 172.21.15.28:0/4045035283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50103260 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 --2- 172.21.15.28:0/4045035283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a500ffe10 0x7f6a50100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 -- 172.21.15.28:0/4045035283 >> 172.21.15.28:0/4045035283 conn(0x7f6a500fb970 msgr2=0x7f6a500fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 -- 172.21.15.28:0/4045035283 shutdown_connections 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.902+0000 7f6a54a35640 1 -- 172.21.15.28:0/4045035283 wait complete. 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 Processor -- start 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 -- start start 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 0x7f6a50197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 0x7f6a5019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5010d7c0 con 0x7f6a50102de0 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5010d640 con 0x7f6a501037a0 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a54a35640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5010d940 con 0x7f6a500ffe10 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48900/0 (socket says 172.21.15.28:48900) 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 0x7f6a50197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 -- 172.21.15.28:0/503001980 learned_addr learned my addr 172.21.15.28:0/503001980 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 0x7f6a5019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 -- 172.21.15.28:0/503001980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 msgr2=0x7f6a50197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 0x7f6a50197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 -- 172.21.15.28:0/503001980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 msgr2=0x7f6a5019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 0x7f6a5019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a47fff640 1 -- 172.21.15.28:0/503001980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5019fb20 con 0x7f6a50102de0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 0x7f6a50197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.903+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 0x7f6a5019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a47fff640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005940 tx=0x7f6a380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a477fe640 1 -- 172.21.15.28:0/503001980 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3803f030 con 0x7f6a50102de0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a477fe640 1 -- 172.21.15.28:0/503001980 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38003ce0 con 0x7f6a50102de0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5019fdb0 con 0x7f6a50102de0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a477fe640 1 -- 172.21.15.28:0/503001980 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38035070 con 0x7f6a50102de0 2024-09-19T09:39:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.904+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a0210 con 0x7f6a50102de0 2024-09-19T09:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 Processor -- start 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a477fe640 1 -- 172.21.15.28:0/503001980 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6a38042030 con 0x7f6a50102de0 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 -- start start 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5648899a12f0 con 0x564889a47780 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564889949270 con 0x564889a426f0 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.907+0000 7f6a577d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56488999d2b0 con 0x564889a41db0 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:56034/0 (socket says 172.21.15.28:56034) 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/1303768876 learned_addr learned my addr 172.21.15.28:0/1303768876 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/1303768876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 msgr2=0x564889a471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a477fe640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 0x7f6a3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/1303768876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 msgr2=0x564889a49c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a49c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/1303768876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564889948340 con 0x564889a41db0 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a477fe640 1 -- 172.21.15.28:0/503001980 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6a380f00e0 con 0x7f6a50102de0 2024-09-19T09:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 0x7f6a3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6a48011810 tx=0x7f6a48011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c79313d3f68b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.908+0000 7f6a44ff9640 1 -- 172.21.15.28:0/1303768876 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48020070 con 0x564889a41db0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a44ff9640 1 -- 172.21.15.28:0/1303768876 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4800fd00 con 0x564889a41db0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a44ff9640 1 -- 172.21.15.28:0/1303768876 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801b8a0 con 0x564889a41db0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 -- 172.21.15.28:0/1303768876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 msgr2=0x564889a421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 0x7f6a3c0a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6a48011490 tx=0x7f6a4800e3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6a48011810 tx=0x7f6a48011cd0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 -- 172.21.15.28:0/1303768876 shutdown_connections 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a49c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 --2- 172.21.15.28:0/1303768876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a421b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 -- 172.21.15.28:0/1303768876 >> 172.21.15.28:0/1303768876 conn(0x564889a39010 msgr2=0x564889a39410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 -- 172.21.15.28:0/1303768876 shutdown_connections 2024-09-19T09:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.909+0000 7f6a577d0740 1 -- 172.21.15.28:0/1303768876 wait complete. 2024-09-19T09:39:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 -- start start 2024-09-19T09:39:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 Processor -- start 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564889a4a6f0 con 0x564889a47780 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56488992a290 con 0x564889a426f0 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564889a4a870 con 0x564889a41db0 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a577d0740 10 client.0 _ll_register_callbacks cb 0x56488995b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a4ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:56050/0 (socket says 172.21.15.28:56050) 2024-09-19T09:39:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a4e59c640 1 -- 172.21.15.28:0/84528854 learned_addr learned my addr 172.21.15.28:0/84528854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.910+0000 7f6a47fff640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 -- 172.21.15.28:0/84528854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 msgr2=0x564889a4c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a4c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 -- 172.21.15.28:0/84528854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 msgr2=0x564889a534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 -- 172.21.15.28:0/84528854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648896b4210 con 0x564889a41db0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a40012d20 tx=0x7f6a4000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb531441a975de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40012110 con 0x564889a41db0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4800fd00 con 0x564889a41db0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mon.1 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6a40004030 con 0x564889a41db0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a40023070 con 0x564889a41db0 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a577d0740 10 client.26287 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.911+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564889780620 con 0x564889a41db0 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mon.1 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4800fd00 con 0x564889a41db0 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mon.1 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f6a40010430 con 0x564889a41db0 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 10 client.26287 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 10 client.26287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 20 client.26287 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889a57c40 0x564889a5a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 4 client.26287 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648899aea20 con 0x564889a57c40 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a47fff640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889a57c40 0x564889a5a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a47fff640 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889a57c40 0x564889a5a0e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79e9f65df6e6116 server_cookie=afce7d38fbc71f00 in_seq=0 out_seq=0 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.912+0000 7f6a35ffb640 10 client.26287 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a35ffb640 1 -- 172.21.15.28:0/84528854 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648899aea20 con 0x564889a57c40 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a35ffb640 10 client.26287 handle_command_reply: tid=0 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 client.26287 shutdown 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 msgr2=0x564889a4bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a40012d20 tx=0x7f6a4000f790 comp rx=0 tx=0).stop 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 shutdown_connections 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a4c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889a57c40 0x564889a5a0e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a35ffb640 0 client.26287 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 --2- 172.21.15.28:0/84528854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a4bfe0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 >> 172.21.15.28:0/84528854 conn(0x564889a39010 msgr2=0x564889a3a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 shutdown_connections 2024-09-19T09:39:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 1 -- 172.21.15.28:0/84528854 wait complete. 2024-09-19T09:39:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.914+0000 7f6a577d0740 20 client.26287 trim_cache size 0 max 16384 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.929+0000 7f6a577d0740 1 Processor -- start 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.929+0000 7f6a577d0740 1 -- start start 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a53410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a6e10 con 0x564889a47780 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a8980 con 0x564889a426f0 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a577d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a74e0 con 0x564889a41db0 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48928/0 (socket says 172.21.15.28:48928) 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/393892126 learned_addr learned my addr 172.21.15.28:0/393892126 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a47fff640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a53410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/393892126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 msgr2=0x564889a3a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a3a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/393892126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 msgr2=0x564889a53410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a53410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 -- 172.21.15.28:0/393892126 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564889a342d0 con 0x564889a47780 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800f790 tx=0x7f6a48026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5124c853c7263e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a37fff640 1 -- 172.21.15.28:0/393892126 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48020040 con 0x564889a47780 2024-09-19T09:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a37fff640 1 -- 172.21.15.28:0/393892126 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4802b040 con 0x564889a47780 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.930+0000 7f6a37fff640 1 -- 172.21.15.28:0/393892126 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801b490 con 0x564889a47780 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 -- 172.21.15.28:0/393892126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 msgr2=0x564889a3a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800f790 tx=0x7f6a48026000 comp rx=0 tx=0).stop 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 -- 172.21.15.28:0/393892126 shutdown_connections 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a426f0 0x564889a53410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a47780 0x564889a3a940 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 --2- 172.21.15.28:0/393892126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a41db0 0x564889a3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 -- 172.21.15.28:0/393892126 >> 172.21.15.28:0/393892126 conn(0x564889a39010 msgr2=0x564889a381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 -- 172.21.15.28:0/393892126 shutdown_connections 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.931+0000 7f6a577d0740 1 -- 172.21.15.28:0/393892126 wait complete. 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 -- start start 2024-09-19T09:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 Processor -- start 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 0x564889a511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 0x564889a521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a82b0 con 0x564889a41db0 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a7bb0 con 0x564889a47780 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501a4df0 con 0x564889a426f0 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a577d0740 10 client.0 _ll_register_callbacks cb 0x7f6a40009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48936/0 (socket says 172.21.15.28:48936) 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 -- 172.21.15.28:0/2143811486 learned_addr learned my addr 172.21.15.28:0/2143811486 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a47fff640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 0x564889a511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 -- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 msgr2=0x564889a511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 0x564889a511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 -- 172.21.15.28:0/2143811486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 msgr2=0x564889a521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 0x564889a521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 0x564889a521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4e59c640 1 -- 172.21.15.28:0/2143811486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564889a52800 con 0x564889a41db0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a4ed9d640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 0x564889a521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.932+0000 7f6a47fff640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 0x564889a511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a4e59c640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6a40010800 tx=0x7f6a40012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1157525cb26377ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40010a70 con 0x564889a41db0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f6a40018040 con 0x564889a41db0 2024-09-19T09:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4802b040 con 0x564889a41db0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a40039050 con 0x564889a41db0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a577d0740 10 client.16755 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564889ac29e0 con 0x564889a41db0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4802b040 con 0x564889a41db0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.933+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f6a400035f0 con 0x564889a41db0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 10 client.16755 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 20 client.16755 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889ac33c0 0x564889ac57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 4 client.16755 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648899a8d70 con 0x564889ac33c0 2024-09-19T09:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a47fff640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889ac33c0 0x564889ac57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a47fff640 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889ac33c0 0x564889ac57e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970ad97b366f76ad server_cookie=ad9f493d328821ea in_seq=0 out_seq=0 2024-09-19T09:39:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:47.934+0000 7f6a35ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 -- 172.21.15.28:0/2166434980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01082a0 msgr2=0x7f8ff010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 --2- 172.21.15.28:0/2166434980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01082a0 0x7f8ff010ec40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009cd0 tx=0x7f8fe802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 -- 172.21.15.28:0/2166434980 shutdown_connections 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 --2- 172.21.15.28:0/2166434980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01036e0 0x7f8ff0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 --2- 172.21.15.28:0/2166434980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01082a0 0x7f8ff010ec40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 --2- 172.21.15.28:0/2166434980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 -- 172.21.15.28:0/2166434980 >> 172.21.15.28:0/2166434980 conn(0x7f8ff00fc7d0 msgr2=0x7f8ff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 -- 172.21.15.28:0/2166434980 shutdown_connections 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.489+0000 7f8ff8643640 1 -- 172.21.15.28:0/2166434980 wait complete. 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 Processor -- start 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 -- start start 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01082a0 0x7f8ff019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01120a0 con 0x7f8ff01036e0 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0111f20 con 0x7f8ff01082a0 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff8643640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0112220 con 0x7f8ff0102d30 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff5bde640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff5bde640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48954/0 (socket says 172.21.15.28:48954) 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff5bde640 1 -- 172.21.15.28:0/3516800707 learned_addr learned my addr 172.21.15.28:0/3516800707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01082a0 0x7f8ff019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 -- 172.21.15.28:0/3516800707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01082a0 msgr2=0x7f8ff019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01082a0 0x7f8ff019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 -- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 msgr2=0x7f8ff0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff63df640 1 -- 172.21.15.28:0/3516800707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff019d4e0 con 0x7f8ff0102d30 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.490+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0198530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9f0 tx=0x7f8fec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8fe77fe640 1 -- 172.21.15.28:0/3516800707 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00ccc0 con 0x7f8ff0102d30 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff019d7d0 con 0x7f8ff0102d30 2024-09-19T09:39:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8fe77fe640 1 -- 172.21.15.28:0/3516800707 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec005020 con 0x7f8ff0102d30 2024-09-19T09:39:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8fe77fe640 1 -- 172.21.15.28:0/3516800707 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0049e0 con 0x7f8ff0102d30 2024-09-19T09:39:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.491+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a50b0 con 0x7f8ff0102d30 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.493+0000 7f8fe77fe640 1 -- 172.21.15.28:0/3516800707 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8fec005560 con 0x7f8ff0102d30 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 Processor -- start 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 -- start start 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cc64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5581e9c20160 con 0x5581e9cbc170 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5581e9bc7720 con 0x5581e9cbb850 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581e9c1c120 con 0x5581e9cc6a30 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cc64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cc64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48958/0 (socket says 172.21.15.28:48958) 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ff63df640 1 -- 172.21.15.28:0/2595877515 learned_addr learned my addr 172.21.15.28:0/2595877515 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.494+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 -- 172.21.15.28:0/2595877515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 msgr2=0x5581e9cbbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cbbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8fe77fe640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 0x7f8fdc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 -- 172.21.15.28:0/2595877515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 msgr2=0x5581e9cc64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cc64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 -- 172.21.15.28:0/2595877515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e9991ba0 con 0x5581e9cc6a30 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8fe77fe640 1 -- 172.21.15.28:0/3516800707 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8fec0cf960 con 0x7f8ff0102d30 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 0x7f8fdc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc8e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00058c0 tx=0x7f8fe000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c554093b0deddd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8fe4ff9640 1 -- 172.21.15.28:0/2595877515 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe001d070 con 0x5581e9cc6a30 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8fe4ff9640 1 -- 172.21.15.28:0/2595877515 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe0010d20 con 0x5581e9cc6a30 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.495+0000 7f8fe4ff9640 1 -- 172.21.15.28:0/2595877515 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0016340 con 0x5581e9cc6a30 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/2595877515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 msgr2=0x5581e9cc8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 0x7f8fdc0a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009430 tx=0x7f8fe8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc8e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00058c0 tx=0x7f8fe000f790 comp rx=0 tx=0).stop 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/2595877515 shutdown_connections 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc8e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cc64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/2595877515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/2595877515 >> 172.21.15.28:0/2595877515 conn(0x5581e9cb7de0 msgr2=0x5581e9cb81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/2595877515 shutdown_connections 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/2595877515 wait complete. 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 -- start start 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.496+0000 7f8ffaa1c740 1 Processor -- start 2024-09-19T09:39:48.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbc170 0x5581e9cc3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5581e9cc9600 con 0x5581e9cbb850 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5581e9ba8900 con 0x5581e9cbc170 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581e9cc9780 con 0x5581e9cc6a30 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ffaa1c740 10 client.0 _ll_register_callbacks cb 0x5581e9ba6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48960/0 (socket says 172.21.15.28:48960) 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 -- 172.21.15.28:0/3412795091 learned_addr learned my addr 172.21.15.28:0/3412795091 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbc170 0x5581e9cc3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 -- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 msgr2=0x5581e9cc43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 -- 172.21.15.28:0/3412795091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbc170 msgr2=0x5581e9cc3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbc170 0x5581e9cc3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 -- 172.21.15.28:0/3412795091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e9be8c50 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8ff63df640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9c0 tx=0x7f8fec098320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cf03083465a4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.497+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec01c070 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe0010d20 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8fec0a4930 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0051d0 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 10 client.16767 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5581e9cd5340 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe0010d20 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f8fec004c50 con 0x5581e9cbb850 2024-09-19T09:39:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 10 client.16767 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 10 client.16767 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 20 client.16767 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9cd67a0 0x5581e9cd8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 4 client.16767 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e9c2d890 con 0x5581e9cd67a0 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.498+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9cd67a0 0x5581e9cd8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.499+0000 7f8fd9ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.499+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9cd67a0 0x5581e9cd8c40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfb36b2f79d3236 server_cookie=be4691f19d17d652 in_seq=0 out_seq=0 2024-09-19T09:39:48.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.614+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/3412795091 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581e9c2d890 con 0x5581e9cd67a0 2024-09-19T09:39:48.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.614+0000 7f8fd9ffb640 10 client.16767 handle_command_reply: tid=0 2024-09-19T09:39:48.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.614+0000 7f8ffaa1c740 1 client.16767 shutdown 2024-09-19T09:39:48.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.616+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 msgr2=0x5581e9cc3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.616+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9c0 tx=0x7f8fec098320 comp rx=0 tx=0).stop 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 shutdown_connections 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbc170 0x5581e9cc3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9cd67a0 0x5581e9cd8c40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8fd9ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/3412795091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbb850 0x5581e9cc3940 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.621+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 >> 172.21.15.28:0/3412795091 conn(0x5581e9cb7de0 msgr2=0x5581e9cbdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:48.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.625+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 shutdown_connections 2024-09-19T09:39:48.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.625+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/3412795091 wait complete. 2024-09-19T09:39:48.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.625+0000 7f8ffaa1c740 20 client.16767 trim_cache size 0 max 16384 2024-09-19T09:39:48.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.661+0000 7f8ffaa1c740 1 Processor -- start 2024-09-19T09:39:48.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.662+0000 7f8ffaa1c740 1 -- start start 2024-09-19T09:39:48.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.663+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.663+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cdea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.663+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.664+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51694/0 (socket says 172.21.15.28:51694) 2024-09-19T09:39:48.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.664+0000 7f8ff63df640 1 -- 172.21.15.28:0/115678034 learned_addr learned my addr 172.21.15.28:0/115678034 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:48.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.664+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.664+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01ad570 con 0x5581e9cbc170 2024-09-19T09:39:48.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.664+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01af0e0 con 0x5581e9cbb850 2024-09-19T09:39:48.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.665+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01adc40 con 0x5581e9cc6a30 2024-09-19T09:39:48.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.668+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cdea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.668+0000 7f8ff63df640 1 -- 172.21.15.28:0/115678034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 msgr2=0x5581e9cc0ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:39:48.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.668+0000 7f8ff63df640 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc0ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.668+0000 7f8ff63df640 1 -- 172.21.15.28:0/115678034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 msgr2=0x5581e9cdea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.668+0000 7f8ff63df640 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cdea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.669+0000 7f8ff63df640 1 -- 172.21.15.28:0/115678034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e9cc28f0 con 0x5581e9cbb850 2024-09-19T09:39:48.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.670+0000 7f8ff63df640 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fec09bc50 tx=0x7f8fec098280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5624ac23ae54f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.671+0000 7f8fdbfff640 1 -- 172.21.15.28:0/115678034 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec01c040 con 0x5581e9cbb850 2024-09-19T09:39:48.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.671+0000 7f8fdbfff640 1 -- 172.21.15.28:0/115678034 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec09bf50 con 0x5581e9cbb850 2024-09-19T09:39:48.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.672+0000 7f8fdbfff640 1 -- 172.21.15.28:0/115678034 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0a4730 con 0x5581e9cbb850 2024-09-19T09:39:48.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.673+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 msgr2=0x5581e9cde510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:48.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.673+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fec09bc50 tx=0x7f8fec098280 comp rx=0 tx=0).stop 2024-09-19T09:39:48.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.675+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 shutdown_connections 2024-09-19T09:39:48.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.675+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cc6a30 0x5581e9cc0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.675+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9cdea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.675+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/115678034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cbb850 0x5581e9cde510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.675+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 >> 172.21.15.28:0/115678034 conn(0x5581e9cb7de0 msgr2=0x5581e9cbdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:48.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.677+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 shutdown_connections 2024-09-19T09:39:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.678+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/115678034 wait complete. 2024-09-19T09:39:48.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.679+0000 7f8ffaa1c740 1 -- start start 2024-09-19T09:39:48.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.679+0000 7f8ffaa1c740 1 Processor -- start 2024-09-19T09:39:48.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cbb850 0x5581e9ccae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9ccb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01aea10 con 0x5581e9cbc170 2024-09-19T09:39:48.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01ae310 con 0x5581e9cc6a30 2024-09-19T09:39:48.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01ab550 con 0x5581e9cbb850 2024-09-19T09:39:48.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.680+0000 7f8ffaa1c740 10 client.0 _ll_register_callbacks cb 0x7f8fec0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:39:48.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.681+0000 7f8ff6be0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.681+0000 7f8ff63df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cbb850 0x5581e9ccae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.681+0000 7f8ff6be0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:51704/0 (socket says 172.21.15.28:51704) 2024-09-19T09:39:48.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.681+0000 7f8ff6be0640 1 -- 172.21.15.28:0/1230472692 learned_addr learned my addr 172.21.15.28:0/1230472692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:39:48.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff5bde640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9ccb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff6be0640 1 -- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cbb850 msgr2=0x5581e9ccae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cbb850 0x5581e9ccae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff6be0640 1 -- 172.21.15.28:0/1230472692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 msgr2=0x5581e9ccb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:39:48.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9ccb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:48.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.682+0000 7f8ff6be0640 1 -- 172.21.15.28:0/1230472692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e9d49260 con 0x5581e9cc6a30 2024-09-19T09:39:48.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.684+0000 7f8ff6be0640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8035000 tx=0x7f8fe80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18457e8a940b9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:39:48.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.685+0000 7f8fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:39:48.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.686+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004250 con 0x5581e9cc6a30 2024-09-19T09:39:48.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.686+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8fe80043f0 con 0x5581e9cc6a30 2024-09-19T09:39:48.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.686+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec09bf50 con 0x5581e9cc6a30 2024-09-19T09:39:48.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.689+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fe80045f0 con 0x5581e9cc6a30 2024-09-19T09:39:48.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.690+0000 7f8ffaa1c740 10 client.26315 fetch_fsmap learned FSMap version 37 2024-09-19T09:39:48.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.690+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5581e9d4a520 con 0x5581e9cc6a30 2024-09-19T09:39:48.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.694+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec09bf50 con 0x5581e9cc6a30 2024-09-19T09:39:48.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.694+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f8fe8030070 con 0x5581e9cc6a30 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 10 client.26315 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 10 client.26315 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 20 client.26315 populate_metadata read hostname 'smithi028' 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9d4ad90 0x5581e9d4d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 4 client.26315 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T09:39:48.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.697+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x5581e9c27be0 con 0x5581e9d4ad90 2024-09-19T09:39:48.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.700+0000 7f8ff63df640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9d4ad90 0x5581e9d4d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:39:48.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.708+0000 7f8fd9ffb640 10 client.26315 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:48.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:39:48.708+0000 7f8ff63df640 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9d4ad90 0x5581e9d4d1b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9b9f741196d054 server_cookie=9a82f8d0af314973 in_seq=0 out_seq=0 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.361+0000 7f6a35ffb640 1 -- 172.21.15.28:0/2143811486 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648899a8d70 con 0x564889ac33c0 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.361+0000 7f6a35ffb640 10 client.16755 handle_command_reply: tid=0 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.361+0000 7f6a577d0740 1 client.16755 shutdown 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 msgr2=0x564889a50ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6a40010800 tx=0x7f6a40012980 comp rx=0 tx=0).stop 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 shutdown_connections 2024-09-19T09:39:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564889a47780 0x564889a521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564889ac33c0 0x564889ac57e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a35ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564889a426f0 0x564889a511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 --2- 172.21.15.28:0/2143811486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564889a41db0 0x564889a50ca0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 >> 172.21.15.28:0/2143811486 conn(0x564889a39010 msgr2=0x564889a393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 shutdown_connections 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 1 -- 172.21.15.28:0/2143811486 wait complete. 2024-09-19T09:39:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.362+0000 7f6a577d0740 20 client.16755 trim_cache size 0 max 16384 2024-09-19T09:39:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:39:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:39:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:39:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 msgr2=0x7f6a3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 0x7f6a3c0a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6a48011490 tx=0x7f6a4800e3b0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 msgr2=0x7f6a50198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005940 tx=0x7f6a380042f0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 shutdown_connections 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f6a3c0a3240 0x7f6a3c0a5700 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a501037a0 0x7f6a5019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a50102de0 0x7f6a50198440 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 --2- 172.21.15.28:0/503001980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a500ffe10 0x7f6a50197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:39:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 >> 172.21.15.28:0/503001980 conn(0x7f6a500fb970 msgr2=0x7f6a501061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:39:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 shutdown_connections 2024-09-19T09:39:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:39:49.363+0000 7f6a54a35640 1 -- 172.21.15.28:0/503001980 wait complete. 2024-09-19T09:39:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:49 smithi038 ceph-mon[35271]: pgmap v553: 97 pgs: 97 active+clean; 628 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-09-19T09:39:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:49 smithi028 ceph-mon[30972]: pgmap v553: 97 pgs: 97 active+clean; 628 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-09-19T09:39:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:49 smithi078 ceph-mon[35619]: pgmap v553: 97 pgs: 97 active+clean; 628 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-09-19T09:39:51.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:51 smithi038 ceph-mon[35271]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-09-19T09:39:51.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:51 smithi028 ceph-mon[30972]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-09-19T09:39:51.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:51 smithi078 ceph-mon[35619]: pgmap v554: 97 pgs: 97 active+clean; 630 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-09-19T09:39:53.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:53 smithi038 ceph-mon[35271]: pgmap v555: 97 pgs: 97 active+clean; 619 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 809 op/s 2024-09-19T09:39:53.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:53 smithi028 ceph-mon[30972]: pgmap v555: 97 pgs: 97 active+clean; 619 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 809 op/s 2024-09-19T09:39:53.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:53 smithi078 ceph-mon[35619]: pgmap v555: 97 pgs: 97 active+clean; 619 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 809 op/s 2024-09-19T09:39:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:55 smithi038 ceph-mon[35271]: pgmap v556: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 1.09k op/s 2024-09-19T09:39:55.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:55 smithi028 ceph-mon[30972]: pgmap v556: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 1.09k op/s 2024-09-19T09:39:55.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:55 smithi078 ceph-mon[35619]: pgmap v556: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 1.09k op/s 2024-09-19T09:39:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:57 smithi038 ceph-mon[35271]: pgmap v557: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 1.15k op/s 2024-09-19T09:39:57.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:57 smithi028 ceph-mon[30972]: pgmap v557: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 1.15k op/s 2024-09-19T09:39:57.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:57 smithi078 ceph-mon[35619]: pgmap v557: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 1.15k op/s 2024-09-19T09:39:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:39:59 smithi038 ceph-mon[35271]: pgmap v558: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 940 op/s 2024-09-19T09:39:59.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:39:59 smithi028 ceph-mon[30972]: pgmap v558: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 940 op/s 2024-09-19T09:39:59.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:39:59 smithi078 ceph-mon[35619]: pgmap v558: 97 pgs: 97 active+clean; 600 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 940 op/s 2024-09-19T09:40:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:39:59.998+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-09-19T09:40:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:39:59.998+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-09-19T09:40:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:39:59.998+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : mds.h(mds.4): Metadata damage detected 2024-09-19T09:40:00.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:39:59.998+0000 7f1886d7f640 -1 log_channel(cluster) log [ERR] : mds.f(mds.3): Metadata damage detected 2024-09-19T09:40:00.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-mon[30972]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-09-19T09:40:00.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-mon[30972]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-09-19T09:40:00.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-mon[30972]: mds.h(mds.4): Metadata damage detected 2024-09-19T09:40:00.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:00 smithi028 ceph-mon[30972]: mds.f(mds.3): Metadata damage detected 2024-09-19T09:40:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:00 smithi078 ceph-mon[35619]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-09-19T09:40:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:00 smithi078 ceph-mon[35619]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-09-19T09:40:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:00 smithi078 ceph-mon[35619]: mds.h(mds.4): Metadata damage detected 2024-09-19T09:40:00.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:00 smithi078 ceph-mon[35619]: mds.f(mds.3): Metadata damage detected 2024-09-19T09:40:00.925 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:00 smithi038 ceph-mon[35271]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-09-19T09:40:00.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:00 smithi038 ceph-mon[35271]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-09-19T09:40:00.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:00 smithi038 ceph-mon[35271]: mds.h(mds.4): Metadata damage detected 2024-09-19T09:40:00.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:00 smithi038 ceph-mon[35271]: mds.f(mds.3): Metadata damage detected 2024-09-19T09:40:01.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:01 smithi038 ceph-mon[35271]: pgmap v559: 97 pgs: 97 active+clean; 602 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 901 op/s 2024-09-19T09:40:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:01 smithi028 ceph-mon[30972]: pgmap v559: 97 pgs: 97 active+clean; 602 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 901 op/s 2024-09-19T09:40:01.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:01 smithi078 ceph-mon[35619]: pgmap v559: 97 pgs: 97 active+clean; 602 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 901 op/s 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 -- 172.21.15.28:0/2466246020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 msgr2=0x7fc960100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 --2- 172.21.15.28:0/2466246020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960100e40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc950009ca0 tx=0x7fc95002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 -- 172.21.15.28:0/2466246020 shutdown_connections 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 --2- 172.21.15.28:0/2466246020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9601025f0 0x7fc960108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 --2- 172.21.15.28:0/2466246020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101c30 0x7fc9601020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 --2- 172.21.15.28:0/2466246020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960100e40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 -- 172.21.15.28:0/2466246020 >> 172.21.15.28:0/2466246020 conn(0x7fc9600fc1c0 msgr2=0x7fc9600fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.455+0000 7fc9648a5640 1 -- 172.21.15.28:0/2466246020 shutdown_connections 2024-09-19T09:40:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 -- 172.21.15.28:0/2466246020 wait complete. 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 Processor -- start 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 -- start start 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 0x7fc960198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 0x7fc96019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.456+0000 7fc9648a5640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010c6a0 con 0x7fc960100a40 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc9648a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010c520 con 0x7fc960101c30 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc9648a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010c820 con 0x7fc9601025f0 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51316/0 (socket says 172.21.15.28:51316) 2024-09-19T09:40:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95e59c640 1 -- 172.21.15.28:0/3418835943 learned_addr learned my addr 172.21.15.28:0/3418835943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 0x7fc960198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 0x7fc96019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 -- 172.21.15.28:0/3418835943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 msgr2=0x7fc96019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 0x7fc96019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 -- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 msgr2=0x7fc960197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 -- 172.21.15.28:0/3418835943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96019fb30 con 0x7fc960101c30 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 0x7fc96019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95e59c640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 0x7fc960198450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc954005fd0 tx=0x7fc95400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc94f7fe640 1 -- 172.21.15.28:0/3418835943 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954009c30 con 0x7fc960101c30 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc94f7fe640 1 -- 172.21.15.28:0/3418835943 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc954009dd0 con 0x7fc960101c30 2024-09-19T09:40:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96019fe20 con 0x7fc960101c30 2024-09-19T09:40:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.457+0000 7fc94f7fe640 1 -- 172.21.15.28:0/3418835943 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954010460 con 0x7fc960101c30 2024-09-19T09:40:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.458+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601a02e0 con 0x7fc960101c30 2024-09-19T09:40:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc94f7fe640 1 -- 172.21.15.28:0/3418835943 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc954004030 con 0x7fc960101c30 2024-09-19T09:40:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 Processor -- start 2024-09-19T09:40:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 -- start start 2024-09-19T09:40:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc94f7fe640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 0x7fc9480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e64a9ae2f0 con 0x55e64aa4edb0 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e64a956270 con 0x55e64aa4f6f0 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.460+0000 7fc967645740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e64a9aa2b0 con 0x55e64aa54780 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 0x7fc9480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51330/0 (socket says 172.21.15.28:51330) 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 -- 172.21.15.28:0/2154350550 learned_addr learned my addr 172.21.15.28:0/2154350550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc94f7fe640 1 -- 172.21.15.28:0/3418835943 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc9540cf480 con 0x7fc960101c30 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 -- 172.21.15.28:0/2154350550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 msgr2=0x55e64aa56c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa56c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 -- 172.21.15.28:0/2154350550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 msgr2=0x55e64aa541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 -- 172.21.15.28:0/2154350550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64a955340 con 0x55e64aa4edb0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 0x7fc9480a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc95800b610 tx=0x7fc95800b5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fa0 tx=0x7fc950031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c47ca620965995 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc94cff9640 1 -- 172.21.15.28:0/2154350550 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950049070 con 0x55e64aa4edb0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc94cff9640 1 -- 172.21.15.28:0/2154350550 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.461+0000 7fc94cff9640 1 -- 172.21.15.28:0/2154350550 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95002fed0 con 0x55e64aa4edb0 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- 172.21.15.28:0/2154350550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 msgr2=0x55e64aa4f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fa0 tx=0x7fc950031800 comp rx=0 tx=0).stop 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- 172.21.15.28:0/2154350550 shutdown_connections 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa56c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 --2- 172.21.15.28:0/2154350550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa4f1b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- 172.21.15.28:0/2154350550 >> 172.21.15.28:0/2154350550 conn(0x55e64aa46010 msgr2=0x55e64aa46410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- 172.21.15.28:0/2154350550 shutdown_connections 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- 172.21.15.28:0/2154350550 wait complete. 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.462+0000 7fc967645740 1 -- start start 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 Processor -- start 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e64aa57570 con 0x55e64aa4edb0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e64a937290 con 0x55e64aa4f6f0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e64aa576f0 con 0x55e64aa54780 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc967645740 10 client.0 _ll_register_callbacks cb 0x55e64a968140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51338/0 (socket says 172.21.15.28:51338) 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 -- 172.21.15.28:0/2549022456 learned_addr learned my addr 172.21.15.28:0/2549022456 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 -- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 msgr2=0x55e64aa60310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa60310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 -- 172.21.15.28:0/2549022456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 msgr2=0x55e64aa592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 -- 172.21.15.28:0/2549022456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64a6c1210 con 0x55e64aa4edb0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc95e59c640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc950031a40 tx=0x7fc95002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9814014236601e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.463+0000 7fc945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950054060 con 0x55e64aa4edb0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc950041040 con 0x55e64aa4edb0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc950049030 con 0x55e64aa4edb0 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 10 client.16785 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e64a78d620 con 0x55e64aa4edb0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fc95004f600 con 0x55e64aa4edb0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 10 client.16785 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 20 client.16785 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aa64720 0x55e64aa66b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 4 client.16785 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.464+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e64a9bba20 con 0x55e64aa64720 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.465+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aa64720 0x55e64aa66b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.465+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aa64720 0x55e64aa66b80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8a1443724c3c37 server_cookie=f1e45f3248be265b in_seq=0 out_seq=0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.465+0000 7fc945ffb640 10 client.16785 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc945ffb640 1 -- 172.21.15.28:0/2549022456 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e64a9bba20 con 0x55e64aa64720 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc945ffb640 10 client.16785 handle_command_reply: tid=0 2024-09-19T09:40:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 client.16785 shutdown 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 msgr2=0x55e64aa58d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc950031a40 tx=0x7fc95002b040 comp rx=0 tx=0).stop 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 shutdown_connections 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4f6f0 0x55e64aa592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aa64720 0x55e64aa66b80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa60310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 --2- 172.21.15.28:0/2549022456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa58d90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc945ffb640 0 client.16785 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 >> 172.21.15.28:0/2549022456 conn(0x55e64aa46010 msgr2=0x55e64aa47820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 shutdown_connections 2024-09-19T09:40:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 1 -- 172.21.15.28:0/2549022456 wait complete. 2024-09-19T09:40:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.466+0000 7fc967645740 20 client.16785 trim_cache size 0 max 16384 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 Processor -- start 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 -- start start 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 0x55e64aa591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a6d30 con 0x55e64aa4f6f0 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a88a0 con 0x55e64aa4edb0 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc967645740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a7400 con 0x55e64aa54780 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 0x55e64aa591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95dd9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51346/0 (socket says 172.21.15.28:51346) 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95dd9b640 1 -- 172.21.15.28:0/1796577013 learned_addr learned my addr 172.21.15.28:0/1796577013 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 -- 172.21.15.28:0/1796577013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 msgr2=0x55e64aa47400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa47400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 -- 172.21.15.28:0/1796577013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 msgr2=0x55e64aa602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95e59c640 1 -- 172.21.15.28:0/1796577013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64aa45e30 con 0x55e64aa4edb0 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.481+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 0x55e64aa591b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc950031ae0 tx=0x7fc950031bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae64b45df1a67bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc947fff640 1 -- 172.21.15.28:0/1796577013 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950054040 con 0x55e64aa4edb0 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc947fff640 1 -- 172.21.15.28:0/1796577013 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc947fff640 1 -- 172.21.15.28:0/1796577013 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950030cd0 con 0x55e64aa4edb0 2024-09-19T09:40:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc967645740 1 -- 172.21.15.28:0/1796577013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 msgr2=0x55e64aa591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc967645740 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 0x55e64aa591b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc950031ae0 tx=0x7fc950031bd0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc967645740 1 -- 172.21.15.28:0/1796577013 shutdown_connections 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.482+0000 7fc967645740 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa54780 0x55e64aa47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4f6f0 0x55e64aa602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 --2- 172.21.15.28:0/1796577013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa4edb0 0x55e64aa591b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- 172.21.15.28:0/1796577013 >> 172.21.15.28:0/1796577013 conn(0x55e64aa46010 msgr2=0x55e64aa451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- 172.21.15.28:0/1796577013 shutdown_connections 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- 172.21.15.28:0/1796577013 wait complete. 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- start start 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 Processor -- start 2024-09-19T09:40:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa4f6f0 0x55e64aa5e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 0x55e64aa5f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a81d0 con 0x55e64aa4edb0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a7ad0 con 0x55e64aa54780 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601a4d10 con 0x55e64aa4f6f0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc967645740 10 client.0 _ll_register_callbacks cb 0x7fc950047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc95e59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51352/0 (socket says 172.21.15.28:51352) 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc95e59c640 1 -- 172.21.15.28:0/1050633682 learned_addr learned my addr 172.21.15.28:0/1050633682 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 0x55e64aa5f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.483+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa4f6f0 0x55e64aa5e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 -- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa4f6f0 msgr2=0x55e64aa5e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa4f6f0 0x55e64aa5e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 -- 172.21.15.28:0/1050633682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 msgr2=0x55e64aa5f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 0x55e64aa5f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 -- 172.21.15.28:0/1050633682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64aa5f710 con 0x55e64aa4edb0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95ed9d640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 0x55e64aa5f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc95e59c640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc95002fc00 tx=0x7fc95002fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbdee17ffbd7a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95005f030 con 0x55e64aa4edb0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc95004a040 con 0x55e64aa4edb0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc950054040 con 0x55e64aa4edb0 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc967645740 10 client.16791 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e64aacf5a0 con 0x55e64aa4edb0 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.484+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950036040 con 0x55e64aa4edb0 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fc950064450 con 0x55e64aa4edb0 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 10 client.16791 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 20 client.16791 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aacff70 0x55e64aad2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 4 client.16791 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e64a9b5d70 con 0x55e64aacff70 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aacff70 0x55e64aad2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc95dd9b640 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aacff70 0x55e64aad2390 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28622d7be9a84691 server_cookie=c25cb902217cd86f in_seq=0 out_seq=0 2024-09-19T09:40:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:02.485+0000 7fc945ffb640 10 client.16791 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc945ffb640 1 -- 172.21.15.28:0/1050633682 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e64a9b5d70 con 0x55e64aacff70 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc945ffb640 10 client.16791 handle_command_reply: tid=0 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 client.16791 shutdown 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 msgr2=0x55e64aa5dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc95002fc00 tx=0x7fc95002fc30 comp rx=0 tx=0).stop 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 shutdown_connections 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e64aa54780 0x55e64aa5f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e64aacff70 0x55e64aad2390 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e64aa4f6f0 0x55e64aa5e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 --2- 172.21.15.28:0/1050633682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e64aa4edb0 0x55e64aa5dbb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc945ffb640 0 client.16791 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 >> 172.21.15.28:0/1050633682 conn(0x55e64aa46010 msgr2=0x55e64aa463f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 shutdown_connections 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 1 -- 172.21.15.28:0/1050633682 wait complete. 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.631+0000 7fc967645740 20 client.16791 trim_cache size 0 max 16384 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:40:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:40:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 msgr2=0x7fc9480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 0x7fc9480a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc95800b610 tx=0x7fc95800b5a0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 msgr2=0x7fc960198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 0x7fc960198450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc954005fd0 tx=0x7fc95400dde0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 shutdown_connections 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc9480a3240 0x7fc9480a5700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc960101c30 0x7fc960198450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601025f0 0x7fc96019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 --2- 172.21.15.28:0/3418835943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960100a40 0x7fc960197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 >> 172.21.15.28:0/3418835943 conn(0x7fc9600fc1c0 msgr2=0x7fc960069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 shutdown_connections 2024-09-19T09:40:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:03.632+0000 7fc9648a5640 1 -- 172.21.15.28:0/3418835943 wait complete. 2024-09-19T09:40:03.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:03 smithi028 ceph-mon[30972]: pgmap v560: 97 pgs: 97 active+clean; 604 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 795 op/s 2024-09-19T09:40:03.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:03 smithi078 ceph-mon[35619]: pgmap v560: 97 pgs: 97 active+clean; 604 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 795 op/s 2024-09-19T09:40:03.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:03 smithi038 ceph-mon[35271]: pgmap v560: 97 pgs: 97 active+clean; 604 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 795 op/s 2024-09-19T09:40:05.728 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: pgmap v561: 97 pgs: 97 active+clean; 607 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 790 op/s 2024-09-19T09:40:05.728 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:40:05.729 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.729 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.729 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.729 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:05 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: pgmap v561: 97 pgs: 97 active+clean; 607 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 790 op/s 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:05 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[69063]: *** Caught signal (Segmentation fault) ** 2024-09-19T09:40:05.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[69063]: in thread 7fa7e8c71640 thread_name:ms_dispatch 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: pgmap v561: 97 pgs: 97 active+clean; 607 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 790 op/s 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:05.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:05 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:06.357 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:05 smithi078 podman[69797]: 2024-09-19 09:40:05.889144116 +0000 UTC m=+0.287708989 container died ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:40:06.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:06 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T09:40:06.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:06 smithi028 ceph-mon[30972]: Unable to set osd_memory_target on smithi038 to 185696358: error parsing value: Value '185696358' is below minimum 939524096 2024-09-19T09:40:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:06 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T09:40:06.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:06 smithi078 ceph-mon[35619]: Unable to set osd_memory_target on smithi038 to 185696358: error parsing value: Value '185696358' is below minimum 939524096 2024-09-19T09:40:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:06 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T09:40:06.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:06 smithi038 ceph-mon[35271]: Unable to set osd_memory_target on smithi038 to 185696358: error parsing value: Value '185696358' is below minimum 939524096 2024-09-19T09:40:07.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:07 smithi028 ceph-mon[30972]: pgmap v562: 97 pgs: 97 active+clean; 607 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-09-19T09:40:07.845 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:07 smithi078 ceph-mon[35619]: pgmap v562: 97 pgs: 97 active+clean; 607 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-09-19T09:40:07.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:07 smithi038 ceph-mon[35271]: pgmap v562: 97 pgs: 97 active+clean; 607 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-09-19T09:40:09.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:09 smithi028 ceph-mon[30972]: pgmap v563: 97 pgs: 97 active+clean; 603 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.2 MiB/s wr, 625 op/s 2024-09-19T09:40:09.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:09 smithi078 ceph-mon[35619]: pgmap v563: 97 pgs: 97 active+clean; 603 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.2 MiB/s wr, 625 op/s 2024-09-19T09:40:09.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:09 smithi038 ceph-mon[35271]: pgmap v563: 97 pgs: 97 active+clean; 603 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.2 MiB/s wr, 625 op/s 2024-09-19T09:40:11.357 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:10 smithi078 podman[69797]: 2024-09-19 09:40:10.934681328 +0000 UTC m=+5.333246201 container remove ced0ca3905c6ee8646e87e00484159097fbee964d36ec190dbd2f5ea700a2ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T09:40:11.357 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:10 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: pgmap v564: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 780 op/s 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:40:11.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:40:11.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:40:11.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:11 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-09-19T09:40:11.857 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:11 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Consumed 23.557s CPU time. 2024-09-19T09:40:11.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: pgmap v564: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 780 op/s 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:40:11.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: pgmap v564: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 780 op/s 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:40:11.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8fd9ffb640 1 -- 172.21.15.28:0/1230472692 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581e9c27be0 con 0x5581e9d4ad90 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8fd9ffb640 10 client.26315 handle_command_reply: tid=0 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 client.26315 shutdown 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 msgr2=0x5581e9d48cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8035000 tx=0x7f8fe80095c0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 shutdown_connections 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5581e9cc6a30 0x5581e9d48cf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5581e9d4ad90 0x5581e9d4d1b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5581e9cbc170 0x5581e9ccb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 37, 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 26, 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "9e53d277": { 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 26, 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:11.944 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 145.8, 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8fd9ffb640 0 client.26315 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 --2- 172.21.15.28:0/1230472692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581e9cbb850 0x5581e9ccae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 >> 172.21.15.28:0/1230472692 conn(0x5581e9cb7de0 msgr2=0x5581e9cc3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 shutdown_connections 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 1 -- 172.21.15.28:0/1230472692 wait complete. 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.940+0000 7f8ffaa1c740 20 client.26315 trim_cache size 0 max 16384 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 msgr2=0x7f8fdc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 0x7f8fdc0a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009430 tx=0x7f8fe8005c50 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 msgr2=0x7f8ff0198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0198530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9f0 tx=0x7f8fec00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 shutdown_connections 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8fdc0a3240 0x7f8fdc0a5700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ff01082a0 0x7f8ff019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff01036e0 0x7f8ff0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 --2- 172.21.15.28:0/3516800707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff0102d30 0x7f8ff0198530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 >> 172.21.15.28:0/3516800707 conn(0x7f8ff00fc7d0 msgr2=0x7f8ff0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 shutdown_connections 2024-09-19T09:40:11.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:11.941+0000 7f8ff8643640 1 -- 172.21.15.28:0/3516800707 wait complete. 2024-09-19T09:40:12.756 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:12 smithi028 ceph-mon[30972]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-19T09:40:12.757 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:12 smithi028 ceph-mon[30972]: Unable to set osd_memory_target on smithi078 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-19T09:40:12.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9e53d277', quiesce took 26.9 seconds. Will release after: -1.1499999999999986 2024-09-19T09:40:12.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:40:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:12 smithi078 ceph-mon[35619]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-19T09:40:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:12 smithi078 ceph-mon[35619]: Unable to set osd_memory_target on smithi078 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-19T09:40:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:12 smithi038 ceph-mon[35271]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-19T09:40:12.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:12 smithi038 ceph-mon[35271]: Unable to set osd_memory_target on smithi078 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-19T09:40:13.047 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:13.719 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:13 smithi028 ceph-mon[30972]: pgmap v565: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 709 op/s 2024-09-19T09:40:13.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- 172.21.15.28:0/1030414274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc100a80 msgr2=0x7fa9bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:13.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 --2- 172.21.15.28:0/1030414274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc100a80 0x7fa9bc100ee0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009930 tx=0x7fa9ac02a420 comp rx=0 tx=0).stop 2024-09-19T09:40:13.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- 172.21.15.28:0/1030414274 shutdown_connections 2024-09-19T09:40:13.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 --2- 172.21.15.28:0/1030414274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101420 0x7fa9bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:13.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 --2- 172.21.15.28:0/1030414274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc100a80 0x7fa9bc100ee0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 --2- 172.21.15.28:0/1030414274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc106a80 0x7fa9bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- 172.21.15.28:0/1030414274 >> 172.21.15.28:0/1030414274 conn(0x7fa9bc0fc7a0 msgr2=0x7fa9bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- 172.21.15.28:0/1030414274 shutdown_connections 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- 172.21.15.28:0/1030414274 wait complete. 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 Processor -- start 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.803+0000 7fa9c433b640 1 -- start start 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 0x7fa9bc0fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc106a80 0x7fa9bc0fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc110ea0 con 0x7fa9bc101420 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc110d20 con 0x7fa9bc100a80 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c433b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc111020 con 0x7fa9bc106a80 2024-09-19T09:40:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52324/0 (socket says 172.21.15.28:52324) 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 -- 172.21.15.28:0/3330197972 learned_addr learned my addr 172.21.15.28:0/3330197972 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c28d8640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc106a80 0x7fa9bc0fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c20d7640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 0x7fa9bc0fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 -- 172.21.15.28:0/3330197972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc106a80 msgr2=0x7fa9bc0fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc106a80 0x7fa9bc0fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 -- 172.21.15.28:0/3330197972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 msgr2=0x7fa9bc0fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 0x7fa9bc0fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 -- 172.21.15.28:0/3330197972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc1a6990 con 0x7fa9bc101420 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c20d7640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 0x7fa9bc0fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9c18d6640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009230 tx=0x7fa9ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.804+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac03f030 con 0x7fa9bc101420 2024-09-19T09:40:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.805+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc1a6c20 con 0x7fa9bc101420 2024-09-19T09:40:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.805+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac003ad0 con 0x7fa9bc101420 2024-09-19T09:40:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.805+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac035070 con 0x7fa9bc101420 2024-09-19T09:40:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.805+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1a7100 con 0x7fa9bc101420 2024-09-19T09:40:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.806+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc100ee0 con 0x7fa9bc101420 2024-09-19T09:40:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.808+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa9ac042030 con 0x7fa9bc101420 2024-09-19T09:40:13.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.808+0000 7fa9b37fe640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 0x7fa9a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:13.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.808+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fa9ac0f01a0 con 0x7fa9bc101420 2024-09-19T09:40:13.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.808+0000 7fa9c20d7640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 0x7fa9a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:13.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.809+0000 7fa9c20d7640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 0x7fa9a80a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005fd0 tx=0x7fa9b800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:13.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:13.812+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9ac0bcef0 con 0x7fa9bc101420 2024-09-19T09:40:13.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:13 smithi078 ceph-mon[35619]: pgmap v565: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 709 op/s 2024-09-19T09:40:13.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:13 smithi038 ceph-mon[35271]: pgmap v565: 97 pgs: 97 active+clean; 603 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 709 op/s 2024-09-19T09:40:14.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.026+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9bc10a020 con 0x7fa9bc101420 2024-09-19T09:40:14.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.027+0000 7fa9b37fe640 1 -- 172.21.15.28:0/3330197972 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11954 (secure 0 0 0) 0x7fa9ac0bc7f0 con 0x7fa9bc101420 2024-09-19T09:40:14.029 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:40:14.029 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-19T09:38:35:591758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:38:35.429459+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:40:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-19T09:40:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 msgr2=0x7fa9a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 0x7fa9a80a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005fd0 tx=0x7fa9b800b5f0 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 msgr2=0x7fa9bc0fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009230 tx=0x7fa9ac004270 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 shutdown_connections 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fa9a80a3240 0x7fa9a80a5700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc106a80 0x7fa9bc0fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9bc101420 0x7fa9bc0fd2f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 --2- 172.21.15.28:0/3330197972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9bc100a80 0x7fa9bc0fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 >> 172.21.15.28:0/3330197972 conn(0x7fa9bc0fc7a0 msgr2=0x7fa9bc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.029+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 shutdown_connections 2024-09-19T09:40:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:14.030+0000 7fa9c433b640 1 -- 172.21.15.28:0/3330197972 wait complete. 2024-09-19T09:40:14.645 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id 9e53d277 --release --await' 2024-09-19T09:40:14.645 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id 9e53d277 --release --await 2024-09-19T09:40:14.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3330197972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3330197972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:14.902 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:14.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3330197972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:15.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 -- 172.21.15.28:0/996767906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4101de0 msgr2=0x7f13f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 --2- 172.21.15.28:0/996767906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4101de0 0x7f13f41021c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a410 comp rx=0 tx=0).stop 2024-09-19T09:40:15.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 -- 172.21.15.28:0/996767906 shutdown_connections 2024-09-19T09:40:15.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 --2- 172.21.15.28:0/996767906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4102790 0x7f13f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 --2- 172.21.15.28:0/996767906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 0x7f13f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 --2- 172.21.15.28:0/996767906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4101de0 0x7f13f41021c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 -- 172.21.15.28:0/996767906 >> 172.21.15.28:0/996767906 conn(0x7f13f40fd8c0 msgr2=0x7f13f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 -- 172.21.15.28:0/996767906 shutdown_connections 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.338+0000 7f13fb155640 1 -- 172.21.15.28:0/996767906 wait complete. 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 Processor -- start 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 -- start start 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4101de0 0x7f13f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 0x7f13f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4111f70 con 0x7f13f4102790 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4111df0 con 0x7f13f4101de0 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13fb155640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41120f0 con 0x7f13f4106f10 2024-09-19T09:40:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13f8ef1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4101de0 0x7f13f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52344/0 (socket says 172.21.15.28:52344) 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13f3fff640 1 -- 172.21.15.28:0/404422894 learned_addr learned my addr 172.21.15.28:0/404422894 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.339+0000 7f13f96f2640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 0x7f13f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 -- 172.21.15.28:0/404422894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 msgr2=0x7f13f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 0x7f13f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 -- 172.21.15.28:0/404422894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4101de0 msgr2=0x7f13f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4101de0 0x7f13f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 -- 172.21.15.28:0/404422894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41a68b0 con 0x7f13f4102790 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f3fff640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d950 tx=0x7f13e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f1ffb640 1 -- 172.21.15.28:0/404422894 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0017070 con 0x7f13f4102790 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f1ffb640 1 -- 172.21.15.28:0/404422894 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e0009c30 con 0x7f13f4102790 2024-09-19T09:40:15.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41a6ba0 con 0x7f13f4102790 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13f1ffb640 1 -- 172.21.15.28:0/404422894 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e000f6b0 con 0x7f13f4102790 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.340+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a7130 con 0x7f13f4102790 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13f1ffb640 1 -- 172.21.15.28:0/404422894 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f13e00109d0 con 0x7f13f4102790 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 Processor -- start 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 -- start start 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac18e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac198d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558fac0f3620 con 0x558fac199240 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13f1ffb640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 0x7f13e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558fac09ae50 con 0x558fac18e980 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13fd52e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fac0ef5e0 con 0x558fac18e060 2024-09-19T09:40:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13f96f2640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 0x7f13e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.343+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52354/0 (socket says 172.21.15.28:52354) 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f3fff640 1 -- 172.21.15.28:0/67028138 learned_addr learned my addr 172.21.15.28:0/67028138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f96f2640 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac18e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f1ffb640 1 -- 172.21.15.28:0/404422894 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f13e00cf3c0 con 0x7f13f4102790 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 -- 172.21.15.28:0/67028138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 msgr2=0x558fac18e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac18e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 -- 172.21.15.28:0/67028138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 msgr2=0x558fac19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 -- 172.21.15.28:0/67028138 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fac08bd40 con 0x558fac18e980 2024-09-19T09:40:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f3fff640 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f96f2640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 0x7f13e40a5780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0098b0 tx=0x7f13ec009d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac198d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005940 tx=0x7f13e8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a310056055d538 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13deffd640 1 -- 172.21.15.28:0/67028138 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8040070 con 0x558fac18e980 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13deffd640 1 -- 172.21.15.28:0/67028138 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80319f0 con 0x558fac18e980 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.344+0000 7f13deffd640 1 -- 172.21.15.28:0/67028138 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8035040 con 0x558fac18e980 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- 172.21.15.28:0/67028138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 msgr2=0x558fac198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac198d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005940 tx=0x7f13e8005e80 comp rx=0 tx=0).stop 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- 172.21.15.28:0/67028138 shutdown_connections 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac198d00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 --2- 172.21.15.28:0/67028138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac18e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- 172.21.15.28:0/67028138 >> 172.21.15.28:0/67028138 conn(0x558fac18a5f0 msgr2=0x558fac18a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- 172.21.15.28:0/67028138 shutdown_connections 2024-09-19T09:40:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- 172.21.15.28:0/67028138 wait complete. 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.345+0000 7f13fd52e740 1 -- start start 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 Processor -- start 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac196b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558fac19be10 con 0x558fac18e980 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558fac07bf10 con 0x558fac199240 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fac19bf90 con 0x558fac18e060 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13fd52e740 10 client.0 _ll_register_callbacks cb 0x558fac09fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52368/0 (socket says 172.21.15.28:52368) 2024-09-19T09:40:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 -- 172.21.15.28:0/544537284 learned_addr learned my addr 172.21.15.28:0/544537284 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f96f2640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 -- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 msgr2=0x558fac1960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 -- 172.21.15.28:0/544537284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 msgr2=0x558fac196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f3fff640 1 -- 172.21.15.28:0/544537284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fabe02580 con 0x558fac18e980 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.346+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13f3fff640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f13e00ccf40 tx=0x7f13e009b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4d4969d8eb480f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00b4060 con 0x558fac18e980 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f13e0098130 con 0x558fac18e980 2024-09-19T09:40:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80319f0 con 0x558fac18e980 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f13e006db40 con 0x558fac18e980 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13fd52e740 10 client.16809 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558fabecc960 con 0x558fac18e980 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80319f0 con 0x558fac18e980 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f13e00860b0 con 0x558fac18e980 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13fd52e740 10 client.16809 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.347+0000 7f13fd52e740 10 client.16809 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13fd52e740 20 client.16809 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac1a8c90 0x558fac1ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13fd52e740 4 client.16809 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fac100b90 con 0x558fac1a8c90 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac1a8c90 0x558fac1ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac1a8c90 0x558fac1ab150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21fe89351407fa2 server_cookie=c3c6fcb5ead94645 in_seq=0 out_seq=0 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.348+0000 7f13d3fff640 10 client.16809 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13d3fff640 1 -- 172.21.15.28:0/544537284 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fac100b90 con 0x558fac1a8c90 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13d3fff640 10 client.16809 handle_command_reply: tid=0 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 client.16809 shutdown 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 msgr2=0x558fac196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f13e00ccf40 tx=0x7f13e009b600 comp rx=0 tx=0).stop 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 shutdown_connections 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac1a8c90 0x558fac1ab150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e980 0x558fac196600 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 --2- 172.21.15.28:0/544537284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13d3fff640 0 client.16809 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.349+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 >> 172.21.15.28:0/544537284 conn(0x558fac18a5f0 msgr2=0x558fac1902d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:15.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.350+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 shutdown_connections 2024-09-19T09:40:15.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.350+0000 7f13fd52e740 1 -- 172.21.15.28:0/544537284 wait complete. 2024-09-19T09:40:15.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.350+0000 7f13fd52e740 20 client.16809 trim_cache size 0 max 16384 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.368+0000 7f13fd52e740 1 Processor -- start 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 -- start start 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac189640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41af630 con 0x558fac199240 2024-09-19T09:40:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41b11a0 con 0x558fac18e980 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13fd52e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41afd00 con 0x558fac18e060 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52384/0 (socket says 172.21.15.28:52384) 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 -- 172.21.15.28:0/2937704139 learned_addr learned my addr 172.21.15.28:0/2937704139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f3fff640 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac189640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 -- 172.21.15.28:0/2937704139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 msgr2=0x558fac1913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 -- 172.21.15.28:0/2937704139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 msgr2=0x558fac189640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac189640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 -- 172.21.15.28:0/2937704139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fac198000 con 0x558fac199240 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.369+0000 7f13f96f2640 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f13ec010ba0 tx=0x7f13ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6caed70ce2c906b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13de7fc640 1 -- 172.21.15.28:0/2937704139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec004b20 con 0x558fac199240 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13de7fc640 1 -- 172.21.15.28:0/2937704139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec004cc0 con 0x558fac199240 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13de7fc640 1 -- 172.21.15.28:0/2937704139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec005780 con 0x558fac199240 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 -- 172.21.15.28:0/2937704139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 msgr2=0x558fac1b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f13ec010ba0 tx=0x7f13ec004ab0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 -- 172.21.15.28:0/2937704139 shutdown_connections 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac18e980 0x558fac189640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac199240 0x558fac1b0a70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 --2- 172.21.15.28:0/2937704139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e060 0x558fac1913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 -- 172.21.15.28:0/2937704139 >> 172.21.15.28:0/2937704139 conn(0x558fac18a5f0 msgr2=0x558fac188f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 -- 172.21.15.28:0/2937704139 shutdown_connections 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.370+0000 7f13fd52e740 1 -- 172.21.15.28:0/2937704139 wait complete. 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 -- start start 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 Processor -- start 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e980 0x558fac19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41b0ad0 con 0x558fac18e060 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41b03d0 con 0x558fac199240 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41ad610 con 0x558fac18e980 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13fd52e740 10 client.0 _ll_register_callbacks cb 0x7f13e0097e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13f8ef1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13f8ef1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52392/0 (socket says 172.21.15.28:52392) 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13f8ef1640 1 -- 172.21.15.28:0/726399699 learned_addr learned my addr 172.21.15.28:0/726399699 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.371+0000 7f13f96f2640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f3fff640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e980 0x558fac19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 -- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e980 msgr2=0x558fac19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e980 0x558fac19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 -- 172.21.15.28:0/726399699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 msgr2=0x558fac19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 -- 172.21.15.28:0/726399699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fac19ef20 con 0x558fac18e060 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13f8ef1640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f13e8031550 tx=0x7f13e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580de80211bf777f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8040040 con 0x558fac18e060 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f13e8031a00 con 0x558fac18e060 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ec004cc0 con 0x558fac18e060 2024-09-19T09:40:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f13e8035040 con 0x558fac18e060 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13fd52e740 10 client.16815 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558fac214a30 con 0x558fac18e060 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.372+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec004cc0 con 0x558fac18e060 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f13e80038c0 con 0x558fac18e060 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 10 client.16815 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 10 client.16815 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 20 client.16815 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac214dc0 0x558fac2171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 4 client.16815 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e53d277", "release": true, "await": true}] 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e53d277", "release": true, "await": true}) -- 0x558fac0fb0a0 con 0x558fac214dc0 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13f3fff640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac214dc0 0x558fac2171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13f3fff640 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac214dc0 0x558fac2171e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b87f5ac2805be43 server_cookie=d2b93479af66c042 in_seq=0 out_seq=0 2024-09-19T09:40:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:15.373+0000 7f13d3fff640 10 client.16815 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:15 smithi028 ceph-mon[30972]: pgmap v566: 97 pgs: 97 active+clean; 606 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.5 MiB/s wr, 762 op/s 2024-09-19T09:40:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:15 smithi078 ceph-mon[35619]: pgmap v566: 97 pgs: 97 active+clean; 606 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.5 MiB/s wr, 762 op/s 2024-09-19T09:40:15.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:15 smithi038 ceph-mon[35271]: pgmap v566: 97 pgs: 97 active+clean; 606 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.5 MiB/s wr, 762 op/s 2024-09-19T09:40:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13d3fff640 1 -- 172.21.15.28:0/726399699 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558fac0fb0a0 con 0x558fac214dc0 2024-09-19T09:40:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13d3fff640 10 client.16815 handle_command_reply: tid=0 2024-09-19T09:40:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 client.16815 shutdown 2024-09-19T09:40:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 msgr2=0x558fac1a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f13e8031550 tx=0x7f13e8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 shutdown_connections 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558fac199240 0x558fac19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x558fac214dc0 0x558fac2171e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fac18e980 0x558fac19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 --2- 172.21.15.28:0/726399699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558fac18e060 0x558fac1a1860 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13d3fff640 0 client.16815 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 >> 172.21.15.28:0/726399699 conn(0x558fac18a5f0 msgr2=0x558fac1968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 shutdown_connections 2024-09-19T09:40:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.619+0000 7f13fd52e740 1 -- 172.21.15.28:0/726399699 wait complete. 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.620+0000 7f13fd52e740 20 client.16815 trim_cache size 0 max 16384 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 37, 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 28, 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "9e53d277": { 2024-09-19T09:40:16.623 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 28, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 145.8, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:16.624 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:40:16.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 msgr2=0x7f13e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 0x7f13e40a5780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0098b0 tx=0x7f13ec009d90 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 msgr2=0x7f13f4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d950 tx=0x7f13e000de10 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 shutdown_connections 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f13e40a32c0 0x7f13e40a5780 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4106f10 0x7f13f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f4102790 0x7f13f4074a30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 --2- 172.21.15.28:0/404422894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f4101de0 0x7f13f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 >> 172.21.15.28:0/404422894 conn(0x7f13f40fd8c0 msgr2=0x7f13f40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 shutdown_connections 2024-09-19T09:40:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:16.621+0000 7f13fb155640 1 -- 172.21.15.28:0/404422894 wait complete. 2024-09-19T09:40:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 -- 172.21.15.28:0/3564864165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc107850 msgr2=0x7f30fc109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 --2- 172.21.15.28:0/3564864165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc107850 0x7f30fc109cc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009be0 tx=0x7f30f402a6c0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 -- 172.21.15.28:0/3564864165 shutdown_connections 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 --2- 172.21.15.28:0/3564864165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc107850 0x7f30fc109cc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 --2- 172.21.15.28:0/3564864165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc104f20 0x7f30fc107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 --2- 172.21.15.28:0/3564864165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 -- 172.21.15.28:0/3564864165 >> 172.21.15.28:0/3564864165 conn(0x7f30fc0fc1f0 msgr2=0x7f30fc0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.734+0000 7f310381e640 1 -- 172.21.15.28:0/3564864165 shutdown_connections 2024-09-19T09:40:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.735+0000 7f310381e640 1 -- 172.21.15.28:0/3564864165 wait complete. 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.735+0000 7f310381e640 1 Processor -- start 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.735+0000 7f310381e640 1 -- start start 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 0x7f30fc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 0x7f30fc19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10d830 con 0x7f30fc107850 2024-09-19T09:40:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10d6b0 con 0x7f30fc104f20 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f310381e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10d9b0 con 0x7f30fc1025f0 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f31015ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 0x7f30fc19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3100db9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 0x7f30fc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f31015ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:32806/0 (socket says 172.21.15.28:32806) 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f31015ba640 1 -- 172.21.15.28:0/3623053682 learned_addr learned my addr 172.21.15.28:0/3623053682 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3100db9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 0x7f30fc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:39042/0 (socket says 172.21.15.28:39042) 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 -- 172.21.15.28:0/3623053682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 msgr2=0x7f30fc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 -- 172.21.15.28:0/3623053682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 msgr2=0x7f30fc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 0x7f30fc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 -- 172.21.15.28:0/3623053682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc19fb70 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 0x7f30fc19f500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30f4034040 tx=0x7f30f40305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.736+0000 7f30f27fc640 1 -- 172.21.15.28:0/3623053682 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4041020 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.737+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19fe00 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.737+0000 7f30f27fc640 1 -- 172.21.15.28:0/3623053682 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f4003e50 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.737+0000 7f30f27fc640 1 -- 172.21.15.28:0/3623053682 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4036040 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.737+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a0260 con 0x7f30fc107850 2024-09-19T09:40:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.739+0000 7f30f27fc640 1 -- 172.21.15.28:0/3623053682 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f30f4044020 con 0x7f30fc107850 2024-09-19T09:40:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 Processor -- start 2024-09-19T09:40:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 -- start start 2024-09-19T09:40:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f30f27fc640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 0x55abd02eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55abd02472f0 con 0x55abd02e39a0 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abd01ef270 con 0x55abd02e9370 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f310668f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abd02432b0 con 0x55abd02e42e0 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f3100db9640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.740+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52420/0 (socket says 172.21.15.28:52420) 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 -- 172.21.15.28:0/3516660887 learned_addr learned my addr 172.21.15.28:0/3516660887 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 0x55abd02eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f30f27fc640 1 -- 172.21.15.28:0/3623053682 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f30f40f0540 con 0x7f30fc107850 2024-09-19T09:40:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f3100db9640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 -- 172.21.15.28:0/3516660887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 msgr2=0x55abd02e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 -- 172.21.15.28:0/3516660887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 msgr2=0x55abd02eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 0x55abd02eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 -- 172.21.15.28:0/3516660887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd01ee340 con 0x55abd02e39a0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 0x55abd02eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f3100db9640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 0x7f30e80a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009820 tx=0x7f30ec009b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f31015ba640 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009e10 tx=0x7f30f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e98d725ae68ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f30e77fe640 1 -- 172.21.15.28:0/3516660887 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8019620 con 0x55abd02e39a0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f30e77fe640 1 -- 172.21.15.28:0/3516660887 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8019ca0 con 0x55abd02e39a0 2024-09-19T09:40:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.741+0000 7f30e77fe640 1 -- 172.21.15.28:0/3516660887 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8018e00 con 0x55abd02e39a0 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- 172.21.15.28:0/3516660887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 msgr2=0x55abd02e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009e10 tx=0x7f30f8011810 comp rx=0 tx=0).stop 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- 172.21.15.28:0/3516660887 shutdown_connections 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e9370 0x55abd02eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 --2- 172.21.15.28:0/3516660887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02e3da0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- 172.21.15.28:0/3516660887 >> 172.21.15.28:0/3516660887 conn(0x55abd02df010 msgr2=0x55abd02df410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- 172.21.15.28:0/3516660887 shutdown_connections 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- 172.21.15.28:0/3516660887 wait complete. 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.742+0000 7f310668f740 1 -- start start 2024-09-19T09:40:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 Processor -- start 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 0x55abd02ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55abd02ec340 con 0x55abd02e39a0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abd01d0290 con 0x55abd02e42e0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abd02ec4c0 con 0x55abd02e9370 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f310668f740 10 client.0 _ll_register_callbacks cb 0x55abd0201140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f3100db9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 0x55abd02ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52436/0 (socket says 172.21.15.28:52436) 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f31015ba640 1 -- 172.21.15.28:0/3232054102 learned_addr learned my addr 172.21.15.28:0/3232054102 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.743+0000 7f3101dbb640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 -- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 msgr2=0x55abd02f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 -- 172.21.15.28:0/3232054102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 msgr2=0x55abd02edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 -- 172.21.15.28:0/3232054102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcff5a210 con 0x55abd02e42e0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f31015ba640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f3100db9640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 0x55abd02ee100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0096c0 tx=0x7f30ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ad4c6a1435bd3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f30e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec011e40 con 0x55abd02e42e0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f30ec012920 con 0x55abd02e42e0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8019ca0 con 0x55abd02e42e0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f30ec012b20 con 0x55abd02e42e0 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f310668f740 10 client.26369 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55abd0026620 con 0x55abd02e42e0 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.744+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8019ca0 con 0x55abd02e42e0 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f30ec020050 con 0x55abd02e42e0 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 10 client.26369 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 10 client.26369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 20 client.26369 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd02f98a0 0x55abd02fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 4 client.26369 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd0254a20 con 0x55abd02f98a0 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f31015ba640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd02f98a0 0x55abd02fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f30e4ff9640 10 client.26369 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.745+0000 7f31015ba640 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd02f98a0 0x55abd02fbcf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ae272ea264d94d server_cookie=fd0dd29a38c7c5de in_seq=0 out_seq=0 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f30e4ff9640 1 -- 172.21.15.28:0/3232054102 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abd0254a20 con 0x55abd02f98a0 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f30e4ff9640 10 client.26369 handle_command_reply: tid=0 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 client.26369 shutdown 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 msgr2=0x55abd02ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 0x55abd02ee100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0096c0 tx=0x7f30ec004730 comp rx=0 tx=0).stop 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 shutdown_connections 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e42e0 0x55abd02ee100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd02f98a0 0x55abd02fbcf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 --2- 172.21.15.28:0/3232054102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e39a0 0x55abd02edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f30e4ff9640 0 client.26369 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 >> 172.21.15.28:0/3232054102 conn(0x55abd02df010 msgr2=0x55abd02e0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 shutdown_connections 2024-09-19T09:40:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 1 -- 172.21.15.28:0/3232054102 wait complete. 2024-09-19T09:40:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.746+0000 7f310668f740 20 client.26369 trim_cache size 0 max 16384 2024-09-19T09:40:16.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:16 smithi028 ceph-mon[30972]: pgmap v567: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-09-19T09:40:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 Processor -- start 2024-09-19T09:40:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 -- start start 2024-09-19T09:40:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1a8fd0 con 0x55abd02df010 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1aab40 con 0x55abd02e39a0 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f310668f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1a96a0 con 0x55abd02e42e0 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f31015ba640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52452/0 (socket says 172.21.15.28:52452) 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f31015ba640 1 -- 172.21.15.28:0/1678967666 learned_addr learned my addr 172.21.15.28:0/1678967666 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.762+0000 7f3100db9640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f3101dbb640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 -- 172.21.15.28:0/1678967666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 msgr2=0x55abd02e0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 -- 172.21.15.28:0/1678967666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 msgr2=0x55abd02f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 -- 172.21.15.28:0/1678967666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd02dee30 con 0x55abd02df010 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f3101dbb640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f3100db9640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f31015ba640 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f30f801b010 tx=0x7f30f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a1cb77e3e7e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f30e6ffd640 1 -- 172.21.15.28:0/1678967666 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f802f020 con 0x55abd02df010 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f30e6ffd640 1 -- 172.21.15.28:0/1678967666 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8005520 con 0x55abd02df010 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.763+0000 7f30e6ffd640 1 -- 172.21.15.28:0/1678967666 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8004b20 con 0x55abd02df010 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- 172.21.15.28:0/1678967666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 msgr2=0x55abd02e3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f30f801b010 tx=0x7f30f8011810 comp rx=0 tx=0).stop 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- 172.21.15.28:0/1678967666 shutdown_connections 2024-09-19T09:40:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e42e0 0x55abd02e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 --2- 172.21.15.28:0/1678967666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02df010 0x55abd02e3d80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- 172.21.15.28:0/1678967666 >> 172.21.15.28:0/1678967666 conn(0x55abd02e8e90 msgr2=0x55abd02de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- 172.21.15.28:0/1678967666 shutdown_connections 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- 172.21.15.28:0/1678967666 wait complete. 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 -- start start 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.764+0000 7f310668f740 1 Processor -- start 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1aa470 con 0x55abd02e42e0 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1a9d70 con 0x55abd02e39a0 2024-09-19T09:40:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1a6fb0 con 0x55abd02e9370 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f310668f740 10 client.0 _ll_register_callbacks cb 0x7f30ec013030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f3100db9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:39086/0 (socket says 172.21.15.28:39086) 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f3100db9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52454/0 (socket says 172.21.15.28:52454) 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 -- 172.21.15.28:0/2005182108 learned_addr learned my addr 172.21.15.28:0/2005182108 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f3101dbb640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 -- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 msgr2=0x55abd02f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 -- 172.21.15.28:0/2005182108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 msgr2=0x55abd02f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 -- 172.21.15.28:0/2005182108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd02f4400 con 0x55abd02e39a0 2024-09-19T09:40:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f3100db9640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f31015ba640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30f8013040 tx=0x7f30f8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1baf73cd48f9fff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f30e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.765+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f802f030 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f30f80186f0 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8005520 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f30f803a070 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 10 client.26375 fetch_fsmap learned FSMap version 37 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55abd0364690 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8005520 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f30f802b650 con 0x55abd02e39a0 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 10 client.26375 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:40:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 20 client.26375 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd0364e80 0x55abd03672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 4 client.26375 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd024ed70 con 0x55abd0364e80 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.766+0000 7f3100db9640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd0364e80 0x55abd03672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.767+0000 7f3100db9640 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd0364e80 0x55abd03672a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5add142ee53d4663 server_cookie=59d087112b666249 in_seq=0 out_seq=0 2024-09-19T09:40:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:16.767+0000 7f30e4ff9640 10 client.26375 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:16.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:16 smithi078 ceph-mon[35619]: pgmap v567: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-09-19T09:40:16.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:16 smithi038 ceph-mon[35271]: pgmap v567: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-09-19T09:40:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.051+0000 7f30e4ff9640 1 -- 172.21.15.28:0/2005182108 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abd024ed70 con 0x55abd0364e80 2024-09-19T09:40:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.051+0000 7f30e4ff9640 10 client.26375 handle_command_reply: tid=0 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.051+0000 7f310668f740 1 client.26375 shutdown 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.051+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 msgr2=0x55abd02f28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.051+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30f8013040 tx=0x7f30f8005dc0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 shutdown_connections 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abd0364e80 0x55abd03672a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f30e4ff9640 0 client.26375 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abd02e9370 0x55abd02f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abd02e42e0 0x55abd02f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 --2- 172.21.15.28:0/2005182108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abd02e39a0 0x55abd02f28a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 >> 172.21.15.28:0/2005182108 conn(0x55abd02e8e90 msgr2=0x55abd02de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 shutdown_connections 2024-09-19T09:40:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 1 -- 172.21.15.28:0/2005182108 wait complete. 2024-09-19T09:40:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.052+0000 7f310668f740 20 client.26375 trim_cache size 0 max 16384 2024-09-19T09:40:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:40:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:40:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:40:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 msgr2=0x7f30e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 0x7f30e80a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009820 tx=0x7f30ec009b50 comp rx=0 tx=0).stop 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 msgr2=0x7f30fc19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 0x7f30fc19f500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30f4034040 tx=0x7f30f40305d0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 shutdown_connections 2024-09-19T09:40:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30e80a3240 0x7f30e80a5700 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30fc104f20 0x7f30fc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30fc107850 0x7f30fc19f500 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 --2- 172.21.15.28:0/3623053682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30fc1025f0 0x7f30fc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 >> 172.21.15.28:0/3623053682 conn(0x7f30fc0fc1f0 msgr2=0x7f30fc102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 shutdown_connections 2024-09-19T09:40:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:17.053+0000 7f310381e640 1 -- 172.21.15.28:0/3623053682 wait complete. 2024-09-19T09:40:17.342 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9e53d277', total seconds elapsed: 31.5 2024-09-19T09:40:17.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T09:40:17.343 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:40:17.672 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:18.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 -- 172.21.15.28:0/950358685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb81036e0 msgr2=0x7f9bb8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 --2- 172.21.15.28:0/950358685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb81036e0 0x7f9bb8107b90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009930 tx=0x7f9ba802a420 comp rx=0 tx=0).stop 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 -- 172.21.15.28:0/950358685 shutdown_connections 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 --2- 172.21.15.28:0/950358685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 0x7f9bb810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 --2- 172.21.15.28:0/950358685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb81036e0 0x7f9bb8107b90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 --2- 172.21.15.28:0/950358685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb8102d30 0x7f9bb8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 -- 172.21.15.28:0/950358685 >> 172.21.15.28:0/950358685 conn(0x7f9bb80fc7d0 msgr2=0x7f9bb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.100+0000 7f9bc006f640 1 -- 172.21.15.28:0/950358685 shutdown_connections 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.101+0000 7f9bc006f640 1 -- 172.21.15.28:0/950358685 wait complete. 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.101+0000 7f9bc006f640 1 Processor -- start 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.101+0000 7f9bc006f640 1 -- start start 2024-09-19T09:40:18.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb81036e0 0x7f9bb8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 0x7f9bb819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81120a0 con 0x7f9bb8102d30 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8111f20 con 0x7f9bb81036e0 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bc006f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8112220 con 0x7f9bb81082a0 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52472/0 (socket says 172.21.15.28:52472) 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 -- 172.21.15.28:0/3869332994 learned_addr learned my addr 172.21.15.28:0/3869332994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbe60c640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 0x7f9bb819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbd60a640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb81036e0 0x7f9bb8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 -- 172.21.15.28:0/3869332994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 msgr2=0x7f9bb819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 0x7f9bb819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 -- 172.21.15.28:0/3869332994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb81036e0 msgr2=0x7f9bb8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb81036e0 0x7f9bb8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 -- 172.21.15.28:0/3869332994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb819d5f0 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.102+0000 7f9bbde0b640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d950 tx=0x7f9bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.103+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4017030 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.103+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb819d8e0 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.103+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb4004020 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.103+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40047a0 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.103+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb81a51c0 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.104+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb8103110 con 0x7f9bb8102d30 2024-09-19T09:40:18.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.106+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9bb40041d0 con 0x7f9bb8102d30 2024-09-19T09:40:18.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.106+0000 7f9baeffd640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 0x7f9ba40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:18.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.106+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f9bb40cf960 con 0x7f9bb8102d30 2024-09-19T09:40:18.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.107+0000 7f9bbd60a640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 0x7f9ba40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:18.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.107+0000 7f9bbd60a640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 0x7f9ba40a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009900 tx=0x7f9ba8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:18.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.110+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9bb409c6b0 con 0x7f9bb8102d30 2024-09-19T09:40:18.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.325+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bb810a2b0 con 0x7f9bb8102d30 2024-09-19T09:40:18.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.326+0000 7f9baeffd640 1 -- 172.21.15.28:0/3869332994 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11954 (secure 0 0 0) 0x7f9bb409bfb0 con 0x7f9bb8102d30 2024-09-19T09:40:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:40:18.329 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-19T09:38:35:591758+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":16536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:38:35.429459+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25804":{"gid":25804,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/686387781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":686387781},{"type":"v1","addr":"172.21.15.78:6833","nonce":686387781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:40:18.329 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 msgr2=0x7f9ba40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 0x7f9ba40a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009900 tx=0x7f9ba8005e50 comp rx=0 tx=0).stop 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 msgr2=0x7f9bb8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d950 tx=0x7f9bb400de10 comp rx=0 tx=0).stop 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 shutdown_connections 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9ba40a3240 0x7f9ba40a5700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bb81036e0 0x7f9bb8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81082a0 0x7f9bb819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 --2- 172.21.15.28:0/3869332994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bb8102d30 0x7f9bb8198640 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:18.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.328+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 >> 172.21.15.28:0/3869332994 conn(0x7f9bb80fc7d0 msgr2=0x7f9bb8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:18.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.329+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 shutdown_connections 2024-09-19T09:40:18.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:18.329+0000 7f9bc006f640 1 -- 172.21.15.28:0/3869332994 wait complete. 2024-09-19T09:40:18.895 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:18 smithi028 ceph-mon[30972]: pgmap v568: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1022 KiB/s wr, 516 op/s 2024-09-19T09:40:18.895 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:18 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3869332994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:18.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:18 smithi038 ceph-mon[35271]: pgmap v568: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1022 KiB/s wr, 516 op/s 2024-09-19T09:40:18.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:18 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3869332994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:19.047 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T09:40:19.047 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T09:40:19.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:18 smithi078 ceph-mon[35619]: pgmap v568: 97 pgs: 97 active+clean; 609 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1022 KiB/s wr, 516 op/s 2024-09-19T09:40:19.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:18 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3869332994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:19.324 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:19.737 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:19 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 -- 172.21.15.28:0/1164466952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102570 msgr2=0x7fc0d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 --2- 172.21.15.28:0/1164466952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102570 0x7fc0d01029b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009890 tx=0x7fc0bc02a400 comp rx=0 tx=0).stop 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 -- 172.21.15.28:0/1164466952 shutdown_connections 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 --2- 172.21.15.28:0/1164466952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102570 0x7fc0d01029b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 --2- 172.21.15.28:0/1164466952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0102ef0 0x7fc0d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 --2- 172.21.15.28:0/1164466952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0100e20 0x7fc0d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 -- 172.21.15.28:0/1164466952 >> 172.21.15.28:0/1164466952 conn(0x7fc0d00fc7d0 msgr2=0x7fc0d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 -- 172.21.15.28:0/1164466952 shutdown_connections 2024-09-19T09:40:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.870+0000 7fc0d7168640 1 -- 172.21.15.28:0/1164466952 wait complete. 2024-09-19T09:40:19.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.871+0000 7fc0d7168640 1 Processor -- start 2024-09-19T09:40:19.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.871+0000 7fc0d7168640 1 -- start start 2024-09-19T09:40:19.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0102570 0x7fc0d006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 0x7fc0d006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0110f50 con 0x7fc0d0102570 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0110dd0 con 0x7fc0d0102ef0 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d7168640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01110d0 con 0x7fc0d0100e20 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d4f04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d4f04640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:32892/0 (socket says 172.21.15.28:32892) 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d4f04640 1 -- 172.21.15.28:0/882557535 learned_addr learned my addr 172.21.15.28:0/882557535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0cffff640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0102570 0x7fc0d006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.872+0000 7fc0d5705640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 0x7fc0d006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 -- 172.21.15.28:0/882557535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 msgr2=0x7fc0d006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 0x7fc0d006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 -- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0102570 msgr2=0x7fc0d006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0102570 0x7fc0d006ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 -- 172.21.15.28:0/882557535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d006d590 con 0x7fc0d0100e20 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d5705640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 0x7fc0d006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400ead0 tx=0x7fc0c400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0cdffb640 1 -- 172.21.15.28:0/882557535 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400bb70 con 0x7fc0d0100e20 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d006d880 con 0x7fc0d0100e20 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d0071870 con 0x7fc0d0100e20 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0cdffb640 1 -- 172.21.15.28:0/882557535 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c400bd10 con 0x7fc0d0100e20 2024-09-19T09:40:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.873+0000 7fc0cdffb640 1 -- 172.21.15.28:0/882557535 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017410 con 0x7fc0d0100e20 2024-09-19T09:40:19.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 Processor -- start 2024-09-19T09:40:19.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 -- start start 2024-09-19T09:40:19.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4497170 0x5618c44a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 0x5618c44a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c43fb160 con 0x5618c44a1a30 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618c43a2720 con 0x5618c4496850 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0d9541740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618c43f7120 con 0x5618c4497170 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.877+0000 7fc0cdffb640 1 -- 172.21.15.28:0/882557535 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc0c4004f10 con 0x7fc0d0100e20 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:39160/0 (socket says 172.21.15.28:39160) 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 -- 172.21.15.28:0/3672268437 learned_addr learned my addr 172.21.15.28:0/3672268437 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0cffff640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 0x5618c44a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4497170 0x5618c44a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 -- 172.21.15.28:0/3672268437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4497170 msgr2=0x5618c44a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4497170 0x5618c44a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 -- 172.21.15.28:0/3672268437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 msgr2=0x5618c44a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 0x5618c44a3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0d5705640 1 -- 172.21.15.28:0/3672268437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c416cba0 con 0x5618c4496850 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.878+0000 7fc0cffff640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 0x5618c44a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:19.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0d5705640 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ee0 tx=0x7fc0c800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d818a5b513eef378 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0baffd640 1 -- 172.21.15.28:0/3672268437 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8011640 con 0x5618c4496850 2024-09-19T09:40:19.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0baffd640 1 -- 172.21.15.28:0/3672268437 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8011cc0 con 0x5618c4496850 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0baffd640 1 -- 172.21.15.28:0/3672268437 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c801be40 con 0x5618c4496850 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0cdffb640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 0x7fc0c00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.879+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 0x7fc0c00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0cdffb640 1 -- 172.21.15.28:0/882557535 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc0c400c310 con 0x7fc0d0100e20 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 -- 172.21.15.28:0/3672268437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 msgr2=0x5618c4496c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ee0 tx=0x7fc0c800f730 comp rx=0 tx=0).stop 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d4f04640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 0x7fc0c00a5800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409a0d0 tx=0x7fc0c4068090 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 -- 172.21.15.28:0/3672268437 shutdown_connections 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c44a1a30 0x5618c44a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4497170 0x5618c44a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 --2- 172.21.15.28:0/3672268437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c4496c30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 -- 172.21.15.28:0/3672268437 >> 172.21.15.28:0/3672268437 conn(0x5618c4492de0 msgr2=0x5618c44931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 -- 172.21.15.28:0/3672268437 shutdown_connections 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.880+0000 7fc0d9541740 1 -- 172.21.15.28:0/3672268437 wait complete. 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 -- start start 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 Processor -- start 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4497170 0x5618c449ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c449f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c44a8a10 con 0x5618c4496850 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618c4383900 con 0x5618c4497170 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5618c44a8b90 con 0x5618c44a1a30 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.881+0000 7fc0d9541740 10 client.0 _ll_register_callbacks cb 0x5618c4381780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52514/0 (socket says 172.21.15.28:52514) 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 -- 172.21.15.28:0/1081504534 learned_addr learned my addr 172.21.15.28:0/1081504534 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c449f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 -- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 msgr2=0x5618c449f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c449f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 -- 172.21.15.28:0/1081504534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4497170 msgr2=0x5618c449ee70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4497170 0x5618c449ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 -- 172.21.15.28:0/1081504534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c43c3c50 con 0x5618c4496850 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.882+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc0c801bae0 tx=0x7fc0c801b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7339a3994d62954f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.883+0000 7fc0affff640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.883+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c802dbf0 con 0x5618c4496850 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.883+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c8011cc0 con 0x5618c4496850 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.883+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc0c802dd90 con 0x5618c4496850 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc0c801bb80 con 0x5618c4496850 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0d9541740 10 client.16851 fetch_fsmap learned FSMap version 38 2024-09-19T09:40:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5618c44a4360 con 0x5618c4496850 2024-09-19T09:40:19.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8011cc0 con 0x5618c4496850 2024-09-19T09:40:19.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fc0c80103d0 con 0x5618c4496850 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0d9541740 10 client.16851 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0d9541740 10 client.16851 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.884+0000 7fc0d9541740 20 client.16851 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.885+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c44a5820 0x5618c44a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.885+0000 7fc0d9541740 4 client.16851 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.885+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c4408890 con 0x5618c44a5820 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.885+0000 7fc0cffff640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c44a5820 0x5618c44a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.886+0000 7fc0affff640 10 client.16851 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.886+0000 7fc0cffff640 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c44a5820 0x5618c44a7c80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892a5c2dd36c45e4 server_cookie=6508b2bda7fc5328 in_seq=0 out_seq=0 2024-09-19T09:40:19.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.889+0000 7fc0affff640 1 -- 172.21.15.28:0/1081504534 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618c4408890 con 0x5618c44a5820 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.889+0000 7fc0affff640 10 client.16851 handle_command_reply: tid=0 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.889+0000 7fc0d9541740 1 client.16851 shutdown 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 msgr2=0x5618c449e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc0c801bae0 tx=0x7fc0c801b9e0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 shutdown_connections 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4497170 0x5618c449ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c44a5820 0x5618c44a7c80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c449f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1081504534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4496850 0x5618c449e930 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 >> 172.21.15.28:0/1081504534 conn(0x5618c4492de0 msgr2=0x5618c4498ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0affff640 0 client.16851 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:19.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 shutdown_connections 2024-09-19T09:40:19.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.890+0000 7fc0d9541740 1 -- 172.21.15.28:0/1081504534 wait complete. 2024-09-19T09:40:19.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.891+0000 7fc0d9541740 20 client.16851 trim_cache size 0 max 16384 2024-09-19T09:40:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.917+0000 7fc0d9541740 1 Processor -- start 2024-09-19T09:40:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 -- start start 2024-09-19T09:40:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4496850 0x5618c449d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c44a1a30 0x5618c449f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b3d00 con 0x5618c4497170 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b58a0 con 0x5618c44a1a30 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d9541740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b43d0 con 0x5618c4496850 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4496850 0x5618c449d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52526/0 (socket says 172.21.15.28:52526) 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 -- 172.21.15.28:0/146053309 learned_addr learned my addr 172.21.15.28:0/146053309 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0d5705640 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c44a1a30 0x5618c449f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 -- 172.21.15.28:0/146053309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4496850 msgr2=0x5618c449d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4496850 0x5618c449d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 -- 172.21.15.28:0/146053309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c44a1a30 msgr2=0x5618c449f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c44a1a30 0x5618c449f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.918+0000 7fc0cffff640 1 -- 172.21.15.28:0/146053309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c4492880 con 0x5618c4497170 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0cffff640 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc000c00 tx=0x7fc0bc030460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec85cdced725d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0ba7fc640 1 -- 172.21.15.28:0/146053309 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc037040 con 0x5618c4497170 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0ba7fc640 1 -- 172.21.15.28:0/146053309 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0bc004430 con 0x5618c4497170 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0ba7fc640 1 -- 172.21.15.28:0/146053309 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0305d0 con 0x5618c4497170 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0d9541740 1 -- 172.21.15.28:0/146053309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 msgr2=0x5618c44992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.919+0000 7fc0d9541740 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc000c00 tx=0x7fc0bc030460 comp rx=0 tx=0).stop 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 -- 172.21.15.28:0/146053309 shutdown_connections 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c44a1a30 0x5618c449f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44992b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 --2- 172.21.15.28:0/146053309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c4496850 0x5618c449d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 -- 172.21.15.28:0/146053309 >> 172.21.15.28:0/146053309 conn(0x5618c4492de0 msgr2=0x5618c44a09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 -- 172.21.15.28:0/146053309 shutdown_connections 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 -- 172.21.15.28:0/146053309 wait complete. 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 -- start start 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 Processor -- start 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.920+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c451a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b51d0 con 0x5618c4497170 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b4ad0 con 0x5618c4496850 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b1ce0 con 0x5618c44a1a30 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 10 client.0 _ll_register_callbacks cb 0x7fc0c8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52538/0 (socket says 172.21.15.28:52538) 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:39190/0 (socket says 172.21.15.28:39190) 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0cffff640 1 -- 172.21.15.28:0/1726894352 learned_addr learned my addr 172.21.15.28:0/1726894352 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c451a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 -- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 msgr2=0x5618c451a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c451a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 -- 172.21.15.28:0/1726894352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 msgr2=0x5618c44b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 -- 172.21.15.28:0/1726894352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c451ac10 con 0x5618c4496850 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0cffff640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d5705640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8010010 tx=0x7fc0c801b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9ad3df532e6737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0affff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80105e0 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0bc004430 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.921+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7fc0c8003970 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc0c8010e60 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 10 client.26393 fetch_fsmap learned FSMap version 38 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5618c451bd90 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0bc004430 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fc0c8011ab0 con 0x5618c4496850 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 10 client.26393 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 10 client.26393 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 20 client.26393 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c451c410 0x5618c451e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 4 client.26393 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T09:40:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x5618c4402be0 con 0x5618c451c410 2024-09-19T09:40:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.922+0000 7fc0cffff640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c451c410 0x5618c451e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.923+0000 7fc0cffff640 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c451c410 0x5618c451e800 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a7117ac83f15a6 server_cookie=501005ac578a4216 in_seq=0 out_seq=0 2024-09-19T09:40:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:19.923+0000 7fc0affff640 10 client.26393 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:19.930 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:19 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:20.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:19 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:20.927 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:20 smithi038 ceph-mon[35271]: pgmap v569: 97 pgs: 97 active+clean; 599 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.1 MiB/s wr, 534 op/s 2024-09-19T09:40:21.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:20 smithi028 ceph-mon[30972]: pgmap v569: 97 pgs: 97 active+clean; 599 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.1 MiB/s wr, 534 op/s 2024-09-19T09:40:21.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:20 smithi078 ceph-mon[35619]: pgmap v569: 97 pgs: 97 active+clean; 599 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.1 MiB/s wr, 534 op/s 2024-09-19T09:40:21.859 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-09-19T09:40:21.860 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 systemd[1]: Stopped Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:40:21.860 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service: Consumed 23.557s CPU time. 2024-09-19T09:40:21.860 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 systemd[1]: Starting Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:40:22.357 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 podman[70004]: 2024-09-19 09:40:21.761026373 +0000 UTC m=+0.016265684 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:40:22.357 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:21 smithi078 podman[70004]: 2024-09-19 09:40:21.905420388 +0000 UTC m=+0.160659693 container create 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:40:22.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:22 smithi038 ceph-mon[35271]: pgmap v570: 97 pgs: 97 active+clean; 596 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 977 KiB/s wr, 435 op/s 2024-09-19T09:40:23.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:22 smithi028 ceph-mon[30972]: pgmap v570: 97 pgs: 97 active+clean; 596 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 977 KiB/s wr, 435 op/s 2024-09-19T09:40:23.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:22 smithi078 ceph-mon[35619]: pgmap v570: 97 pgs: 97 active+clean; 596 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 977 KiB/s wr, 435 op/s 2024-09-19T09:40:24.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:24 smithi078 podman[70004]: 2024-09-19 09:40:24.332327437 +0000 UTC m=+2.587566733 container init 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T09:40:24.608 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:24 smithi078 podman[70004]: 2024-09-19 09:40:24.335650097 +0000 UTC m=+2.590889393 container start 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-19T09:40:24.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:24 smithi038 ceph-mon[35271]: pgmap v571: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 531 op/s 2024-09-19T09:40:25.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:24 smithi028 ceph-mon[30972]: pgmap v571: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 531 op/s 2024-09-19T09:40:25.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:24 smithi078 ceph-mon[35619]: pgmap v571: 97 pgs: 97 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 531 op/s 2024-09-19T09:40:25.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:25 smithi078 bash[70004]: 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b 2024-09-19T09:40:25.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:25 smithi078 systemd[1]: Started Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:40:25.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:25 smithi078 ceph-mds[70022]: -- 172.21.15.78:0/1118083052 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c0571d860 con 0x560c056fb400 2024-09-19T09:40:25.607 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:40:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: starting mds.l at 2024-09-19T09:40:25.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:25 smithi038 ceph-mon[35271]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:40:25.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:25 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:40:26.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:25 smithi028 ceph-mon[30972]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:40:26.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:25 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:40:26.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:25 smithi078 ceph-mon[35619]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-09-19T09:40:26.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:25 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-19T09:40:26.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0affff640 1 -- 172.21.15.28:0/1726894352 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5618c4402be0 con 0x5618c451c410 2024-09-19T09:40:26.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0affff640 10 client.26393 handle_command_reply: tid=0 2024-09-19T09:40:26.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 client.26393 shutdown 2024-09-19T09:40:26.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 msgr2=0x5618c44b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8010010 tx=0x7fc0c801b9e0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 shutdown_connections 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5618c451c410 0x5618c451e800 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618c44a1a30 0x5618c451a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c4497170 0x5618c44b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 --2- 172.21.15.28:0/1726894352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5618c4496850 0x5618c44b6840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0affff640 0 client.26393 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.595+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 >> 172.21.15.28:0/1726894352 conn(0x5618c4492de0 msgr2=0x5618c44b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.596+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 shutdown_connections 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.596+0000 7fc0d9541740 1 -- 172.21.15.28:0/1726894352 wait complete. 2024-09-19T09:40:26.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.596+0000 7fc0d9541740 20 client.26393 trim_cache size 0 max 16384 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 39, 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 30, 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "c2823cd6": { 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 30, 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:40:26.599 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 145.8, 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "QUIESCED", 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:26.601 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:26.601 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:26.601 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:26.601 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:40:26.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.596+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 msgr2=0x7fc0c00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.596+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 0x7fc0c00a5800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409a0d0 tx=0x7fc0c4068090 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 msgr2=0x7fc0d0073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400ead0 tx=0x7fc0c400ef90 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 shutdown_connections 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fc0c00a3340 0x7fc0c00a5800 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0d0102ef0 0x7fc0d006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0102570 0x7fc0d006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 --2- 172.21.15.28:0/882557535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0d0100e20 0x7fc0d0073a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 >> 172.21.15.28:0/882557535 conn(0x7fc0d00fc7d0 msgr2=0x7fc0d0104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 shutdown_connections 2024-09-19T09:40:26.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:26.597+0000 7fc0d7168640 1 -- 172.21.15.28:0/882557535 wait complete. 2024-09-19T09:40:27.315 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c2823cd6', quiesce took 10.0 seconds. Will release after: 15.75 2024-09-19T09:40:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:27 smithi078 ceph-mon[35619]: pgmap v572: 97 pgs: 97 active+clean; 581 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-19T09:40:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:27 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6832/2606259516,v1:172.21.15.78:6833/2606259516] up:boot 2024-09-19T09:40:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:27 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:27.608 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:27 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:40:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:27 smithi038 ceph-mon[35271]: pgmap v572: 97 pgs: 97 active+clean; 581 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-19T09:40:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:27 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6832/2606259516,v1:172.21.15.78:6833/2606259516] up:boot 2024-09-19T09:40:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:27 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:27.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:27 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:40:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:27 smithi028 ceph-mon[30972]: pgmap v572: 97 pgs: 97 active+clean; 581 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-19T09:40:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:27 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6832/2606259516,v1:172.21.15.78:6833/2606259516] up:boot 2024-09-19T09:40:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:27 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:27.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:27 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:40:29.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:29 smithi028 ceph-mon[30972]: pgmap v573: 97 pgs: 97 active+clean; 580 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 816 KiB/s wr, 272 op/s 2024-09-19T09:40:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:29 smithi078 ceph-mon[35619]: pgmap v573: 97 pgs: 97 active+clean; 580 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 816 KiB/s wr, 272 op/s 2024-09-19T09:40:29.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:29 smithi038 ceph-mon[35271]: pgmap v573: 97 pgs: 97 active+clean; 580 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 816 KiB/s wr, 272 op/s 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 -- 172.21.15.28:0/3199749056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc1025f0 msgr2=0x7f7cbc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/3199749056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc108e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009ca0 tx=0x7f7cb402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 -- 172.21.15.28:0/3199749056 shutdown_connections 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/3199749056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc101c30 0x7f7cbc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/3199749056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc108e80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/3199749056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc100a40 0x7f7cbc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 -- 172.21.15.28:0/3199749056 >> 172.21.15.28:0/3199749056 conn(0x7f7cbc0fc1c0 msgr2=0x7f7cbc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 -- 172.21.15.28:0/3199749056 shutdown_connections 2024-09-19T09:40:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.146+0000 7f7cc0f98640 1 -- 172.21.15.28:0/3199749056 wait complete. 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 Processor -- start 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 -- start start 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 0x7f7cbc06a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc10c6a0 con 0x7f7cbc101c30 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc10c520 con 0x7f7cbc100a40 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cc0f98640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc10c820 con 0x7f7cbc1025f0 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cb9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cb9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35078/0 (socket says 172.21.15.28:35078) 2024-09-19T09:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/2389926313 learned_addr learned my addr 172.21.15.28:0/2389926313 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.147+0000 7f7cba59c640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 0x7f7cbc06a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/2389926313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc1025f0 msgr2=0x7f7cbc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/2389926313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 msgr2=0x7f7cbc06a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 0x7f7cbc06a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/2389926313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc19f980 con 0x7f7cbc101c30 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cba59c640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 0x7f7cbc06a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9f0 tx=0x7f7cb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cab7fe640 1 -- 172.21.15.28:0/2389926313 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000cd80 con 0x7f7cbc101c30 2024-09-19T09:40:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc19fc70 con 0x7f7cbc101c30 2024-09-19T09:40:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cab7fe640 1 -- 172.21.15.28:0/2389926313 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb0010910 con 0x7f7cbc101c30 2024-09-19T09:40:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cab7fe640 1 -- 172.21.15.28:0/2389926313 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb00188f0 con 0x7f7cbc101c30 2024-09-19T09:40:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.148+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc1a01c0 con 0x7f7cbc101c30 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 Processor -- start 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cab7fe640 1 -- 172.21.15.28:0/2389926313 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7cb0010ac0 con 0x7f7cbc101c30 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 -- start start 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b6e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7aca2f0 con 0x55b9e7b669a0 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7a72270 con 0x55b9e7b672e0 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7ac62b0 con 0x55b9e7b6c370 2024-09-19T09:40:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35084/0 (socket says 172.21.15.28:35084) 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cbad9d640 1 -- 172.21.15.28:0/1632291815 learned_addr learned my addr 172.21.15.28:0/1632291815 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b6e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.151+0000 7f7cba59c640 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 -- 172.21.15.28:0/1632291815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 msgr2=0x55b9e7b6e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b6e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 -- 172.21.15.28:0/1632291815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 msgr2=0x55b9e7b6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 -- 172.21.15.28:0/1632291815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e7a71340 con 0x55b9e7b669a0 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cab7fe640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 0x7f7ca40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cab7fe640 1 -- 172.21.15.28:0/2389926313 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f7cb00cfd40 con 0x7f7cbc101c30 2024-09-19T09:40:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cba59c640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 0x7f7ca40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009ca0 tx=0x7f7cb4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b716c14f9185adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/1632291815 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4031d70 con 0x55b9e7b669a0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/1632291815 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb4036040 con 0x55b9e7b669a0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.152+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/1632291815 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4004320 con 0x55b9e7b669a0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1632291815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 msgr2=0x55b9e7b66da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cba59c640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 0x7f7ca40a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55b9e7b67d10 tx=0x7f7cac0084c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009ca0 tx=0x7f7cb4009800 comp rx=0 tx=0).stop 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1632291815 shutdown_connections 2024-09-19T09:40:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b6e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1632291815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b66da0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1632291815 >> 172.21.15.28:0/1632291815 conn(0x55b9e7b62010 msgr2=0x55b9e7b62410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1632291815 shutdown_connections 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.153+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1632291815 wait complete. 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 -- start start 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 Processor -- start 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b669a0 0x55b9e7b70960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7b6f100 con 0x55b9e7b6c370 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7a53290 con 0x55b9e7b672e0 2024-09-19T09:40:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9e7b6f280 con 0x55b9e7b669a0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cc3e10740 10 client.0 _ll_register_callbacks cb 0x55b9e7a84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35094/0 (socket says 172.21.15.28:35094) 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cbad9d640 1 -- 172.21.15.28:0/501088142 learned_addr learned my addr 172.21.15.28:0/501088142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.154+0000 7f7cba59c640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b669a0 0x55b9e7b70960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 -- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b669a0 msgr2=0x55b9e7b70960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b669a0 0x55b9e7b70960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 -- 172.21.15.28:0/501088142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 msgr2=0x55b9e7b70ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 -- 172.21.15.28:0/501088142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e77dd210 con 0x55b9e7b6c370 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402a900 tx=0x7f7cb402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f232ce7c025cee6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40317a0 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f7cb40040a0 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb4036040 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7cb40042a0 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cc3e10740 10 client.16875 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b9e78a9620 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb4036040 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.155+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7cb404d070 con 0x55b9e7b6c370 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 10 client.16875 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 20 client.16875 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7b7c360 0x55b9e7b7e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 4 client.16875 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9e7ad7a20 con 0x55b9e7b7c360 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cba59c640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7b7c360 0x55b9e7b7e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7ca1ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.156+0000 7f7cba59c640 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7b7c360 0x55b9e7b7e750 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2788040a0762f4bd server_cookie=dda2762c33726232 in_seq=0 out_seq=0 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/501088142 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9e7ad7a20 con 0x55b9e7b7c360 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7ca1ffb640 10 client.16875 handle_command_reply: tid=0 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 client.16875 shutdown 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 msgr2=0x55b9e7b77ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402a900 tx=0x7f7cb402f310 comp rx=0 tx=0).stop 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 shutdown_connections 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7b7c360 0x55b9e7b7e750 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b6c370 0x55b9e7b77ee0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/501088142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b669a0 0x55b9e7b70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 >> 172.21.15.28:0/501088142 conn(0x55b9e7b62010 msgr2=0x55b9e7b63820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7ca1ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 shutdown_connections 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 1 -- 172.21.15.28:0/501088142 wait complete. 2024-09-19T09:40:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.157+0000 7f7cc3e10740 20 client.16875 trim_cache size 0 max 16384 2024-09-19T09:40:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 Processor -- start 2024-09-19T09:40:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 -- start start 2024-09-19T09:40:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1a8dc0 con 0x55b9e7b669a0 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1aa960 con 0x55b9e7b672e0 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1a9490 con 0x55b9e7b6c370 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35110/0 (socket says 172.21.15.28:35110) 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 -- 172.21.15.28:0/3162835218 learned_addr learned my addr 172.21.15.28:0/3162835218 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 -- 172.21.15.28:0/3162835218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 msgr2=0x55b9e7b70ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 -- 172.21.15.28:0/3162835218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 msgr2=0x55b9e7b70960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 -- 172.21.15.28:0/3162835218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e7b78290 con 0x55b9e7b669a0 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.172+0000 7f7cba59c640 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009010 tx=0x7f7cac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e65699fafa022 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7ca3fff640 1 -- 172.21.15.28:0/3162835218 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004220 con 0x55b9e7b669a0 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7ca3fff640 1 -- 172.21.15.28:0/3162835218 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac0043c0 con 0x55b9e7b669a0 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7ca3fff640 1 -- 172.21.15.28:0/3162835218 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0035f0 con 0x55b9e7b669a0 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 -- 172.21.15.28:0/3162835218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 msgr2=0x55b9e7b6bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009010 tx=0x7f7cac014000 comp rx=0 tx=0).stop 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 -- 172.21.15.28:0/3162835218 shutdown_connections 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b672e0 0x55b9e7b70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7b70ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/3162835218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b669a0 0x55b9e7b6bae0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 -- 172.21.15.28:0/3162835218 >> 172.21.15.28:0/3162835218 conn(0x55b9e7b62010 msgr2=0x55b9e7b684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 -- 172.21.15.28:0/3162835218 shutdown_connections 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.173+0000 7f7cc3e10740 1 -- 172.21.15.28:0/3162835218 wait complete. 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 -- start start 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 Processor -- start 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b669a0 0x55b9e7be2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7be96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1aa260 con 0x55b9e7b672e0 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1a9b60 con 0x55b9e7b669a0 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1a6da0 con 0x55b9e7b6c370 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cc3e10740 10 client.0 _ll_register_callbacks cb 0x7f7cac008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35116/0 (socket says 172.21.15.28:35116) 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/1499379247 learned_addr learned my addr 172.21.15.28:0/1499379247 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cba59c640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b669a0 0x55b9e7be2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cbad9d640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7be96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 msgr2=0x55b9e7be96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7be96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/1499379247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b669a0 msgr2=0x55b9e7be2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b669a0 0x55b9e7be2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 -- 172.21.15.28:0/1499379247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e7be9d00 con 0x55b9e7b672e0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7cb9d9b640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0098d60 tx=0x7f7cb0098630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdecaac9a1d1de76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.174+0000 7f7ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb009bcf0 con 0x55b9e7b672e0 2024-09-19T09:40:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f7cb009b9e0 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac0043c0 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7cb009ca90 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 10 client.16881 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b9e7beb0e0 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac0043c0 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7cb00980e0 con 0x55b9e7b672e0 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 10 client.16881 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 20 client.16881 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7beba70 0x55b9e7bede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 4 client.16881 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:40:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9e7ad1d70 con 0x55b9e7beba70 2024-09-19T09:40:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.175+0000 7f7cba59c640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7beba70 0x55b9e7bede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.176+0000 7f7ca1ffb640 10 client.16881 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.176+0000 7f7cba59c640 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7beba70 0x55b9e7bede90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a291616766b7c1 server_cookie=ea8b4c5794589a83 in_seq=0 out_seq=0 2024-09-19T09:40:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7ca1ffb640 1 -- 172.21.15.28:0/1499379247 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9e7ad1d70 con 0x55b9e7beba70 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7ca1ffb640 10 client.16881 handle_command_reply: tid=0 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 client.16881 shutdown 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 msgr2=0x55b9e7be26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0098d60 tx=0x7f7cb0098630 comp rx=0 tx=0).stop 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 shutdown_connections 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55b9e7beba70 0x55b9e7bede90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9e7b6c370 0x55b9e7be96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7ca1ffb640 0 client.16881 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b9e7b672e0 0x55b9e7be26b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 --2- 172.21.15.28:0/1499379247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b9e7b669a0 0x55b9e7be2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 >> 172.21.15.28:0/1499379247 conn(0x55b9e7b62010 msgr2=0x55b9e7b70900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 shutdown_connections 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 1 -- 172.21.15.28:0/1499379247 wait complete. 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.220+0000 7f7cc3e10740 20 client.16881 trim_cache size 0 max 16384 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:40:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 msgr2=0x7f7ca40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 0x7f7ca40a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55b9e7b67d10 tx=0x7f7cac0084c0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 msgr2=0x7f7cbc19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9f0 tx=0x7f7cb000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 shutdown_connections 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7ca40a3240 0x7f7ca40a5700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc1025f0 0x7f7cbc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cbc101c30 0x7f7cbc19ca70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 --2- 172.21.15.28:0/2389926313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cbc100a40 0x7f7cbc06a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 >> 172.21.15.28:0/2389926313 conn(0x7f7cbc0fc1c0 msgr2=0x7f7cbc104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 shutdown_connections 2024-09-19T09:40:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:30.221+0000 7f7cc0f98640 1 -- 172.21.15.28:0/2389926313 wait complete. 2024-09-19T09:40:31.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:31 smithi078 ceph-mon[35619]: pgmap v574: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 823 KiB/s wr, 291 op/s 2024-09-19T09:40:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:31 smithi038 ceph-mon[35271]: pgmap v574: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 823 KiB/s wr, 291 op/s 2024-09-19T09:40:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:31 smithi028 ceph-mon[30972]: pgmap v574: 97 pgs: 97 active+clean; 577 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 823 KiB/s wr, 291 op/s 2024-09-19T09:40:33.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:33 smithi078 ceph-mon[35619]: pgmap v575: 97 pgs: 97 active+clean; 576 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 545 KiB/s wr, 200 op/s 2024-09-19T09:40:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:33 smithi038 ceph-mon[35271]: pgmap v575: 97 pgs: 97 active+clean; 576 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 545 KiB/s wr, 200 op/s 2024-09-19T09:40:33.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:33 smithi028 ceph-mon[30972]: pgmap v575: 97 pgs: 97 active+clean; 576 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 545 KiB/s wr, 200 op/s 2024-09-19T09:40:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:35 smithi078 ceph-mon[35619]: pgmap v576: 97 pgs: 97 active+clean; 572 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 423 KiB/s wr, 153 op/s 2024-09-19T09:40:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:35 smithi038 ceph-mon[35271]: pgmap v576: 97 pgs: 97 active+clean; 572 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 423 KiB/s wr, 153 op/s 2024-09-19T09:40:35.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:35 smithi028 ceph-mon[30972]: pgmap v576: 97 pgs: 97 active+clean; 572 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 423 KiB/s wr, 153 op/s 2024-09-19T09:40:37.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:37 smithi078 ceph-mon[35619]: pgmap v577: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 32 KiB/s wr, 59 op/s 2024-09-19T09:40:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:37 smithi038 ceph-mon[35271]: pgmap v577: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 32 KiB/s wr, 59 op/s 2024-09-19T09:40:37.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:37 smithi028 ceph-mon[30972]: pgmap v577: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 32 KiB/s wr, 59 op/s 2024-09-19T09:40:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:39 smithi078 ceph-mon[35619]: pgmap v578: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-19T09:40:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:39 smithi038 ceph-mon[35271]: pgmap v578: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-19T09:40:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:39 smithi028 ceph-mon[30972]: pgmap v578: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-09-19T09:40:41.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:41 smithi078 ceph-mon[35619]: pgmap v579: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 33 KiB/s wr, 44 op/s 2024-09-19T09:40:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:41 smithi038 ceph-mon[35271]: pgmap v579: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 33 KiB/s wr, 44 op/s 2024-09-19T09:40:41.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:41 smithi028 ceph-mon[30972]: pgmap v579: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 33 KiB/s wr, 44 op/s 2024-09-19T09:40:43.066 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:40:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 -- 172.21.15.28:0/1420448720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8101b50 msgr2=0x7f02e8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 --2- 172.21.15.28:0/1420448720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8101b50 0x7f02e8101f50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009900 tx=0x7f02d802a410 comp rx=0 tx=0).stop 2024-09-19T09:40:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 -- 172.21.15.28:0/1420448720 shutdown_connections 2024-09-19T09:40:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 --2- 172.21.15.28:0/1420448720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8102d40 0x7f02e81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 --2- 172.21.15.28:0/1420448720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8103700 0x7f02e8109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 --2- 172.21.15.28:0/1420448720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8101b50 0x7f02e8101f50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 -- 172.21.15.28:0/1420448720 >> 172.21.15.28:0/1420448720 conn(0x7f02e80fd2f0 msgr2=0x7f02e80ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 -- 172.21.15.28:0/1420448720 shutdown_connections 2024-09-19T09:40:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.319+0000 7f02ece5c640 1 -- 172.21.15.28:0/1420448720 wait complete. 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 Processor -- start 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 -- start start 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8101b50 0x7f02e8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8103700 0x7f02e819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d7b0 con 0x7f02e8102d40 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d630 con 0x7f02e8103700 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02ece5c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d930 con 0x7f02e8101b50 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34308/0 (socket says 172.21.15.28:34308) 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02e659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8101b50 0x7f02e8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02e5d9b640 1 -- 172.21.15.28:0/2536327045 learned_addr learned my addr 172.21.15.28:0/2536327045 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.320+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8103700 0x7f02e819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 -- 172.21.15.28:0/2536327045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8101b50 msgr2=0x7f02e8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8101b50 0x7f02e8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 -- 172.21.15.28:0/2536327045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8103700 msgr2=0x7f02e819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8103700 0x7f02e819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 -- 172.21.15.28:0/2536327045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e819fb70 con 0x7f02e8102d40 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02d77fe640 1 -- 172.21.15.28:0/2536327045 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00cd80 con 0x7f02e8102d40 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02d77fe640 1 -- 172.21.15.28:0/2536327045 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02dc010910 con 0x7f02e8102d40 2024-09-19T09:40:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e819fe60 con 0x7f02e8102d40 2024-09-19T09:40:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02d77fe640 1 -- 172.21.15.28:0/2536327045 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0188f0 con 0x7f02e8102d40 2024-09-19T09:40:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.321+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81a0320 con 0x7f02e8102d40 2024-09-19T09:40:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 Processor -- start 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 -- start start 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02d77fe640 1 -- 172.21.15.28:0/2536327045 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f02dc010ac0 con 0x7f02e8102d40 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 0x564ce15d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15d3780 0x564ce15d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564ce152d2f0 con 0x564ce15cddb0 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564ce14d5270 con 0x564ce15ce6f0 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02efbf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ce15292b0 con 0x564ce15d3780 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02e6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02e6d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34320/0 (socket says 172.21.15.28:34320) 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02e6d9d640 1 -- 172.21.15.28:0/1915049708 learned_addr learned my addr 172.21.15.28:0/1915049708 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.324+0000 7f02e659c640 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 0x564ce15d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15d3780 0x564ce15d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 -- 172.21.15.28:0/1915049708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15d3780 msgr2=0x564ce15d5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15d3780 0x564ce15d5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 -- 172.21.15.28:0/1915049708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 msgr2=0x564ce15ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 -- 172.21.15.28:0/1915049708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce14d4340 con 0x564ce15ce6f0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02e659c640 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 0x564ce15d31b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1a85c45220f607 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02d4ff9640 1 -- 172.21.15.28:0/1915049708 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8040070 con 0x564ce15ce6f0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02d4ff9640 1 -- 172.21.15.28:0/1915049708 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d80319f0 con 0x564ce15ce6f0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02d4ff9640 1 -- 172.21.15.28:0/1915049708 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8035040 con 0x564ce15ce6f0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.325+0000 7f02d77fe640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 0x7f02d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 0x7f02d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02d77fe640 1 -- 172.21.15.28:0/2536327045 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f02dc0cfc90 con 0x7f02e8102d40 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 -- 172.21.15.28:0/1915049708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 msgr2=0x564ce15d31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 0x7f02d00a5700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e00096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 0x564ce15d31b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d8005c00 comp rx=0 tx=0).stop 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 -- 172.21.15.28:0/1915049708 shutdown_connections 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15ce6f0 0x564ce15d31b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15d3780 0x564ce15d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1915049708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 -- 172.21.15.28:0/1915049708 >> 172.21.15.28:0/1915049708 conn(0x564ce15c5010 msgr2=0x564ce15c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 -- 172.21.15.28:0/1915049708 shutdown_connections 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.326+0000 7f02efbf3740 1 -- 172.21.15.28:0/1915049708 wait complete. 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 -- start start 2024-09-19T09:40:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 Processor -- start 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564ce15d66f0 con 0x564ce15cddb0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564ce14b6290 con 0x564ce15d3780 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ce15d6870 con 0x564ce15ce6f0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02efbf3740 10 client.0 _ll_register_callbacks cb 0x564ce14e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34336/0 (socket says 172.21.15.28:34336) 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 -- 172.21.15.28:0/2264241896 learned_addr learned my addr 172.21.15.28:0/2264241896 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 -- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 msgr2=0x564ce15d8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15d8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 -- 172.21.15.28:0/2264241896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 msgr2=0x564ce15df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15df4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e659c640 1 -- 172.21.15.28:0/2264241896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce1240210 con 0x564ce15cddb0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.327+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02e659c640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f02d80057c0 tx=0x7f02d8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d35103438a1fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8040030 con 0x564ce15cddb0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d80319f0 con 0x564ce15cddb0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f02d80317f0 con 0x564ce15cddb0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f02d8035070 con 0x564ce15cddb0 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02efbf3740 10 client.16893 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ce130c620 con 0x564ce15cddb0 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.328+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d8005340 con 0x564ce15cddb0 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f02d80037b0 con 0x564ce15cddb0 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 10 client.16893 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 20 client.16893 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce15e37c0 0x564ce15e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 4 client.16893 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f02d80319f0 con 0x564ce15e37c0 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce15e37c0 0x564ce15e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce15e37c0 0x564ce15e5bd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fb5172ab54805d server_cookie=8f163ff7bf2c825b in_seq=0 out_seq=0 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.329+0000 7f02cdffb640 10 client.16893 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02cdffb640 1 -- 172.21.15.28:0/2264241896 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f02d80319f0 con 0x564ce15e37c0 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02cdffb640 10 client.16893 handle_command_reply: tid=0 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 client.16893 shutdown 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 msgr2=0x564ce15d7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f02d80057c0 tx=0x7f02d8005250 comp rx=0 tx=0).stop 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 shutdown_connections 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce15e37c0 0x564ce15e5bd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 --2- 172.21.15.28:0/2264241896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15d7f40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02cdffb640 0 client.16893 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 >> 172.21.15.28:0/2264241896 conn(0x564ce15c5010 msgr2=0x564ce15c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.330+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 shutdown_connections 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.331+0000 7f02efbf3740 1 -- 172.21.15.28:0/2264241896 wait complete. 2024-09-19T09:40:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.331+0000 7f02efbf3740 20 client.16893 trim_cache size 0 max 16384 2024-09-19T09:40:43.346 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 Processor -- start 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 -- start start 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15cddb0 0x564ce15c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a6e10 con 0x564ce15ce6f0 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a89b0 con 0x564ce15d3780 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02efbf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a74e0 con 0x564ce15cddb0 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34352/0 (socket says 172.21.15.28:34352) 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 -- 172.21.15.28:0/1666658645 learned_addr learned my addr 172.21.15.28:0/1666658645 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e659c640 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15cddb0 0x564ce15c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 -- 172.21.15.28:0/1666658645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15cddb0 msgr2=0x564ce15c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15cddb0 0x564ce15c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 -- 172.21.15.28:0/1666658645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 msgr2=0x564ce15c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.349+0000 7f02e5d9b640 1 -- 172.21.15.28:0/1666658645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce15d7a40 con 0x564ce15ce6f0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f02dc098d60 tx=0x7f02dc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b5edd5ed52813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02cffff640 1 -- 172.21.15.28:0/1666658645 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0a4f50 con 0x564ce15ce6f0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02cffff640 1 -- 172.21.15.28:0/1666658645 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02dc09a430 con 0x564ce15ce6f0 2024-09-19T09:40:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02cffff640 1 -- 172.21.15.28:0/1666658645 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc09b7e0 con 0x564ce15ce6f0 2024-09-19T09:40:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02efbf3740 1 -- 172.21.15.28:0/1666658645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 msgr2=0x564ce15df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.350+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f02dc098d60 tx=0x7f02dc01f000 comp rx=0 tx=0).stop 2024-09-19T09:40:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- 172.21.15.28:0/1666658645 shutdown_connections 2024-09-19T09:40:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15ce6f0 0x564ce15df3e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- 172.21.15.28:0/1666658645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15cddb0 0x564ce15c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- 172.21.15.28:0/1666658645 >> 172.21.15.28:0/1666658645 conn(0x564ce15c5010 msgr2=0x564ce15d7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- 172.21.15.28:0/1666658645 shutdown_connections 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- 172.21.15.28:0/1666658645 wait complete. 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- start start 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 Processor -- start 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a82b0 con 0x564ce15cddb0 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a7bb0 con 0x564ce15d3780 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a4df0 con 0x564ce15ce6f0 2024-09-19T09:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02efbf3740 10 client.0 _ll_register_callbacks cb 0x7f02e0009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.351+0000 7f02e659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34354/0 (socket says 172.21.15.28:34354) 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:35956/0 (socket says 172.21.15.28:35956) 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 -- 172.21.15.28:0/284979589 learned_addr learned my addr 172.21.15.28:0/284979589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e6d9d640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 -- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 msgr2=0x564ce15dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 -- 172.21.15.28:0/284979589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 msgr2=0x564ce15dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 -- 172.21.15.28:0/284979589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce15de480 con 0x564ce15cddb0 2024-09-19T09:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02e659c640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02d8003bd0 tx=0x7f02d8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a1c0b26e60dd4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8040040 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02dc09a430 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f02d8004450 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.352+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f02d80052c0 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 10 client.16899 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564ce164e500 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02dc09a430 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f02d8061030 con 0x564ce15cddb0 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 10 client.16899 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 20 client.16899 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce164ed30 0x564ce1651120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 4 client.16899 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ce153aa20 con 0x564ce164ed30 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.353+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce164ed30 0x564ce1651120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.354+0000 7f02e5d9b640 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce164ed30 0x564ce1651120 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881b6b2f5fc8a163 server_cookie=884f5016fceb4303 in_seq=0 out_seq=0 2024-09-19T09:40:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.354+0000 7f02cdffb640 10 client.16899 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02cdffb640 1 -- 172.21.15.28:0/284979589 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ce153aa20 con 0x564ce164ed30 2024-09-19T09:40:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02cdffb640 10 client.16899 handle_command_reply: tid=0 2024-09-19T09:40:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 client.16899 shutdown 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 msgr2=0x564ce15dc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02d8003bd0 tx=0x7f02d8031a20 comp rx=0 tx=0).stop 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 shutdown_connections 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ce15d3780 0x564ce15dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x564ce164ed30 0x564ce1651120 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce15ce6f0 0x564ce15dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 --2- 172.21.15.28:0/284979589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564ce15cddb0 0x564ce15dc920 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02cdffb640 0 client.16899 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 >> 172.21.15.28:0/284979589 conn(0x564ce15c5010 msgr2=0x564ce15c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 shutdown_connections 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 1 -- 172.21.15.28:0/284979589 wait complete. 2024-09-19T09:40:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.371+0000 7f02efbf3740 20 client.16899 trim_cache size 0 max 16384 2024-09-19T09:40:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:40:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:40:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:40:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 msgr2=0x7f02d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 0x7f02d00a5700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e00096a0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 msgr2=0x7f02e8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 shutdown_connections 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f02d00a3240 0x7f02d00a5700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02e8103700 0x7f02e819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e8102d40 0x7f02e8198490 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 --2- 172.21.15.28:0/2536327045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02e8101b50 0x7f02e8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 >> 172.21.15.28:0/2536327045 conn(0x7f02e80fd2f0 msgr2=0x7f02e81061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 shutdown_connections 2024-09-19T09:40:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:43.372+0000 7f02ece5c640 1 -- 172.21.15.28:0/2536327045 wait complete. 2024-09-19T09:40:43.473 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:43 smithi028 ceph-mon[30972]: pgmap v580: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-19T09:40:43.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:43 smithi078 ceph-mon[35619]: pgmap v580: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-19T09:40:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:43 smithi038 ceph-mon[35271]: pgmap v580: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-19T09:40:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 -- 172.21.15.28:0/2213198855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c10bfc0 msgr2=0x7f8d9c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 --2- 172.21.15.28:0/2213198855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c10e460 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009ca0 tx=0x7f8d9402a950 comp rx=0 tx=0).stop 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 -- 172.21.15.28:0/2213198855 shutdown_connections 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 --2- 172.21.15.28:0/2213198855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c10e460 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 --2- 172.21.15.28:0/2213198855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 0x7f8d9c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 --2- 172.21.15.28:0/2213198855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c102b90 0x7f8d9c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 -- 172.21.15.28:0/2213198855 >> 172.21.15.28:0/2213198855 conn(0x7f8d9c0fc610 msgr2=0x7f8d9c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 -- 172.21.15.28:0/2213198855 shutdown_connections 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.850+0000 7f8da099e640 1 -- 172.21.15.28:0/2213198855 wait complete. 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 Processor -- start 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 -- start start 2024-09-19T09:40:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 0x7f8d9c100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c111dc0 con 0x7f8d9c102b90 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c111c40 con 0x7f8d9c10bfc0 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8da099e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c111f40 con 0x7f8d9c1034b0 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8d9b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8d9b7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34372/0 (socket says 172.21.15.28:34372) 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8d9b7fe640 1 -- 172.21.15.28:0/3489051838 learned_addr learned my addr 172.21.15.28:0/3489051838 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8d9affd640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 0x7f8d9c100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.851+0000 7f8d9bfff640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 -- 172.21.15.28:0/3489051838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 msgr2=0x7f8d9c100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 0x7f8d9c100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 -- 172.21.15.28:0/3489051838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c10bfc0 msgr2=0x7f8d9c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 -- 172.21.15.28:0/3489051838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c1aabf0 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d9b7fe640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d950 tx=0x7f8d8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c017070 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8c009c30 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c1aae20 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00f6b0 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.852+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c1ab280 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.853+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c06a850 con 0x7f8d9c102b90 2024-09-19T09:40:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.854+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8d8c0109d0 con 0x7f8d9c102b90 2024-09-19T09:40:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.855+0000 7f8d98ff9640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 0x7f8d840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.855+0000 7f8d9affd640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 0x7f8d840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.855+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8d8c0cf7d0 con 0x7f8d9c102b90 2024-09-19T09:40:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.856+0000 7f8d9affd640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 0x7f8d840a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c101fb0 tx=0x7f8d9000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:43.859+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8d8c009de0 con 0x7f8d9c102b90 2024-09-19T09:40:44.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.070+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d9c107030 con 0x7f8d9c102b90 2024-09-19T09:40:44.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.071+0000 7f8d98ff9640 1 -- 172.21.15.28:0/3489051838 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11958 (secure 0 0 0) 0x7f8d8c09be70 con 0x7f8d9c102b90 2024-09-19T09:40:44.073 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:40:44.074 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"btime":"2024-09-19T09:40:26:282969+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26392,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2606259516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2606259516},{"type":"v1","addr":"172.21.15.78:6833","nonce":2606259516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:40:25.280731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_16536":{"gid":16536,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6837/4201169804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4201169804},{"type":"v1","addr":"172.21.15.28:6837","nonce":4201169804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24478,24472,24455]},"id":1}]} 2024-09-19T09:40:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 40 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 msgr2=0x7f8d840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 0x7f8d840a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c101fb0 tx=0x7f8d9000a250 comp rx=0 tx=0).stop 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 msgr2=0x7f8d9c1007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d950 tx=0x7f8d8c00de10 comp rx=0 tx=0).stop 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 shutdown_connections 2024-09-19T09:40:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8d840a3240 0x7f8d840a5700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d9c10bfc0 0x7f8d9c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d9c1034b0 0x7f8d9c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 --2- 172.21.15.28:0/3489051838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d9c102b90 0x7f8d9c1007f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 >> 172.21.15.28:0/3489051838 conn(0x7f8d9c0fc610 msgr2=0x7f8d9c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 shutdown_connections 2024-09-19T09:40:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:44.073+0000 7f8da099e640 1 -- 172.21.15.28:0/3489051838 wait complete. 2024-09-19T09:40:44.512 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:44 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3489051838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:44.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:44 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3489051838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:44.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:44 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3489051838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:40:45.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --set-id c2823cd6 --release --await' 2024-09-19T09:40:45.043 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --set-id c2823cd6 --release --await 2024-09-19T09:40:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:40:45.585 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:45 smithi028 ceph-mon[30972]: pgmap v581: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T09:40:45.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:45 smithi078 ceph-mon[35619]: pgmap v581: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T09:40:45.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:45 smithi038 ceph-mon[35271]: pgmap v581: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T09:40:45.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 -- 172.21.15.28:0/717059157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110100a80 msgr2=0x7f3110100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 --2- 172.21.15.28:0/717059157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110100a80 0x7f3110100ee0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009930 tx=0x7f30f802a420 comp rx=0 tx=0).stop 2024-09-19T09:40:45.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 -- 172.21.15.28:0/717059157 shutdown_connections 2024-09-19T09:40:45.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 --2- 172.21.15.28:0/717059157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110101420 0x7f311010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 --2- 172.21.15.28:0/717059157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110100a80 0x7f3110100ee0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 --2- 172.21.15.28:0/717059157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 0x7f3110106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 -- 172.21.15.28:0/717059157 >> 172.21.15.28:0/717059157 conn(0x7f31100fc7a0 msgr2=0x7f31100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:45.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 -- 172.21.15.28:0/717059157 shutdown_connections 2024-09-19T09:40:45.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.783+0000 7f31150f3640 1 -- 172.21.15.28:0/717059157 wait complete. 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 Processor -- start 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 -- start start 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 0x7f311019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 0x7f311019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3110110e80 con 0x7f3110101420 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3110110d00 con 0x7f3110106a80 2024-09-19T09:40:45.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f31150f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3110111000 con 0x7f3110100a80 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f310659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f310659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34386/0 (socket says 172.21.15.28:34386) 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f310659c640 1 -- 172.21.15.28:0/1245234093 learned_addr learned my addr 172.21.15.28:0/1245234093 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f310f59e640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 0x7f311019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.784+0000 7f310ed9d640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 0x7f311019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 -- 172.21.15.28:0/1245234093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 msgr2=0x7f311019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 0x7f311019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 -- 172.21.15.28:0/1245234093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 msgr2=0x7f311019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 0x7f311019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 -- 172.21.15.28:0/1245234093 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311019f630 con 0x7f3110101420 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310ed9d640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 0x7f311019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310659c640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009230 tx=0x7f30f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310cd99640 1 -- 172.21.15.28:0/1245234093 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f803f030 con 0x7f3110101420 2024-09-19T09:40:45.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310cd99640 1 -- 172.21.15.28:0/1245234093 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8003ad0 con 0x7f3110101420 2024-09-19T09:40:45.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311019f8c0 con 0x7f3110101420 2024-09-19T09:40:45.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f310cd99640 1 -- 172.21.15.28:0/1245234093 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8035070 con 0x7f3110101420 2024-09-19T09:40:45.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.785+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101a7200 con 0x7f3110101420 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.787+0000 7f310cd99640 1 -- 172.21.15.28:0/1245234093 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f30f8042030 con 0x7f3110101420 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 Processor -- start 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 -- start start 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x563331451440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x56333145e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5633313b6620 con 0x563331451980 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56333135de50 con 0x563331451060 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f31174cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5633313b25e0 con 0x56333145c240 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f310ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f310ed9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34394/0 (socket says 172.21.15.28:34394) 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f310ed9d640 1 -- 172.21.15.28:0/4004050917 learned_addr learned my addr 172.21.15.28:0/4004050917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f310f59e640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x563331451440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.788+0000 7f310659c640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x56333145e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 -- 172.21.15.28:0/4004050917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 msgr2=0x56333145e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310cd99640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 0x7f30fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x56333145e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 -- 172.21.15.28:0/4004050917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 msgr2=0x563331451440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x563331451440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 -- 172.21.15.28:0/4004050917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333134ed40 con 0x563331451980 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310659c640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x56333145e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310f59e640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x563331451440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310cd99640 1 -- 172.21.15.28:0/1245234093 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f30f80efda0 con 0x7f3110101420 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310f59e640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 0x7f30fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310ed9d640 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f310000bcc0 tx=0x7f310000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7810d8a0d3bac4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310559a640 1 -- 172.21.15.28:0/4004050917 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100015070 con 0x563331451980 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310559a640 1 -- 172.21.15.28:0/4004050917 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310001a940 con 0x563331451980 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.789+0000 7f310559a640 1 -- 172.21.15.28:0/4004050917 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100011a80 con 0x563331451980 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- 172.21.15.28:0/4004050917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 msgr2=0x56333145bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f310000bcc0 tx=0x7f310000f730 comp rx=0 tx=0).stop 2024-09-19T09:40:45.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f310f59e640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 0x7f30fc0a5700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f310800a2c0 tx=0x7f3108004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- 172.21.15.28:0/4004050917 shutdown_connections 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x56333145e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x56333145bd00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 --2- 172.21.15.28:0/4004050917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x563331451440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- 172.21.15.28:0/4004050917 >> 172.21.15.28:0/4004050917 conn(0x56333144d5f0 msgr2=0x56333144d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- 172.21.15.28:0/4004050917 shutdown_connections 2024-09-19T09:40:45.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- 172.21.15.28:0/4004050917 wait complete. 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.790+0000 7f31174cc740 1 -- start start 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 Processor -- start 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 0x563331459110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451980 0x563331459650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56333145ee10 con 0x56333145c240 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56333133ef10 con 0x563331451980 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56333145ef90 con 0x563331451060 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f31174cc740 10 client.0 _ll_register_callbacks cb 0x563331362db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34402/0 (socket says 172.21.15.28:34402) 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 -- 172.21.15.28:0/290808142 learned_addr learned my addr 172.21.15.28:0/290808142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310659c640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451980 0x563331459650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310ed9d640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 0x563331459110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 -- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 msgr2=0x563331459110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 0x563331459110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 -- 172.21.15.28:0/290808142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451980 msgr2=0x563331459650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451980 0x563331459650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310f59e640 1 -- 172.21.15.28:0/290808142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633310c5580 con 0x56333145c240 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.791+0000 7f310ed9d640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 0x563331459110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f310f59e640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f310800fe50 tx=0x7f3108004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cb1692af4278f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108026470 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f310801d3f0 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310001a940 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3108026610 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 10 client.16917 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56333118f960 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310001a940 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f310801db20 con 0x56333145c240 2024-09-19T09:40:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 10 client.16917 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 10 client.16917 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 20 client.16917 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56333146bdf0 0x56333146e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 4 client.16917 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.792+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633313c3b90 con 0x56333146bdf0 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.793+0000 7f310ed9d640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56333146bdf0 0x56333146e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.793+0000 7f310ed9d640 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56333146bdf0 0x56333146e280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8adf92c48cf4b86 server_cookie=f7486c4d0a5e850b in_seq=0 out_seq=0 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.793+0000 7f30f67fc640 10 client.16917 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f30f67fc640 1 -- 172.21.15.28:0/290808142 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633313c3b90 con 0x56333146bdf0 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f30f67fc640 10 client.16917 handle_command_reply: tid=0 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 client.16917 shutdown 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 msgr2=0x563331459b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f310800fe50 tx=0x7f3108004ab0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 shutdown_connections 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451980 0x563331459650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x56333146bdf0 0x56333146e280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331459b90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 --2- 172.21.15.28:0/290808142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451060 0x563331459110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 >> 172.21.15.28:0/290808142 conn(0x56333144d5f0 msgr2=0x563331453320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.794+0000 7f30f67fc640 0 client.16917 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.795+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 shutdown_connections 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.795+0000 7f31174cc740 1 -- 172.21.15.28:0/290808142 wait complete. 2024-09-19T09:40:45.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.795+0000 7f31174cc740 20 client.16917 trim_cache size 0 max 16384 2024-09-19T09:40:45.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.811+0000 7f31174cc740 1 Processor -- start 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.811+0000 7f31174cc740 1 -- start start 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x56333144c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451980 0x563331473b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31101af780 con 0x56333145c240 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31101b1320 con 0x563331451060 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f31174cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31101afe50 con 0x563331451980 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34404/0 (socket says 172.21.15.28:34404) 2024-09-19T09:40:45.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 -- 172.21.15.28:0/2790047364 learned_addr learned my addr 172.21.15.28:0/2790047364 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310659c640 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451980 0x563331473b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310ed9d640 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x56333144c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 -- 172.21.15.28:0/2790047364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451980 msgr2=0x563331473b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451980 0x563331473b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 -- 172.21.15.28:0/2790047364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 msgr2=0x56333144c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x56333144c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 -- 172.21.15.28:0/2790047364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633314580c0 con 0x56333145c240 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.812+0000 7f310f59e640 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f310800a510 tx=0x7f310800a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938685747b2f940b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f3104d99640 1 -- 172.21.15.28:0/2790047364 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080269b0 con 0x56333145c240 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f3104d99640 1 -- 172.21.15.28:0/2790047364 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3108021070 con 0x56333145c240 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f3104d99640 1 -- 172.21.15.28:0/2790047364 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310801ae60 con 0x56333145c240 2024-09-19T09:40:45.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 -- 172.21.15.28:0/2790047364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 msgr2=0x563331474090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f310800a510 tx=0x7f310800a9c0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 -- 172.21.15.28:0/2790047364 shutdown_connections 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56333145c240 0x563331474090 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563331451980 0x563331473b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 --2- 172.21.15.28:0/2790047364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x56333144c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 -- 172.21.15.28:0/2790047364 >> 172.21.15.28:0/2790047364 conn(0x56333144d5f0 msgr2=0x56333144bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.813+0000 7f31174cc740 1 -- 172.21.15.28:0/2790047364 shutdown_connections 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 -- 172.21.15.28:0/2790047364 wait complete. 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 -- start start 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 Processor -- start 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x5633314601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x563331460c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31101b0c20 con 0x563331451980 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f31101b0520 con 0x563331451060 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31101ad760 con 0x56333145c240 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f31174cc740 10 client.0 _ll_register_callbacks cb 0x7f310800fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x5633314601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310659c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34406/0 (socket says 172.21.15.28:34406) 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x5633314601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:34688/0 (socket says 172.21.15.28:34688) 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310659c640 1 -- 172.21.15.28:0/1951297268 learned_addr learned my addr 172.21.15.28:0/1951297268 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.814+0000 7f310f59e640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x563331460c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 -- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 msgr2=0x563331460c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x563331460c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 -- 172.21.15.28:0/1951297268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 msgr2=0x5633314601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x5633314601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 -- 172.21.15.28:0/1951297268 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633314d5d20 con 0x563331451980 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f310659c640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f30f80b8dc0 tx=0x7f30f8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594b445fd3c045f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8049040 con 0x563331451980 2024-09-19T09:40:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f30f8003c80 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3108021070 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f30f80bcaf0 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f31174cc740 10 client.16923 fetch_fsmap learned FSMap version 40 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5633314d7040 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3108021070 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.815+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f30f8031bd0 con 0x563331451980 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 10 client.16923 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T09:40:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 10 client.16923 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 20 client.16923 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5633314d7760 0x5633314d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 4 client.16923 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2823cd6", "release": true, "await": true}] 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2823cd6", "release": true, "await": true}) -- 0x5633313be0a0 con 0x5633314d7760 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f310ed9d640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5633314d7760 0x5633314d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f30f67fc640 10 client.16923 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:45.816+0000 7f310ed9d640 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5633314d7760 0x5633314d9b80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29cb97034349e5d server_cookie=1efde8a46371ce7e in_seq=0 out_seq=0 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f30f67fc640 1 -- 172.21.15.28:0/1951297268 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5633313be0a0 con 0x5633314d7760 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f30f67fc640 10 client.16923 handle_command_reply: tid=0 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 client.16923 shutdown 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 msgr2=0x563331460730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f30f80b8dc0 tx=0x7f30f8005e50 comp rx=0 tx=0).stop 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 shutdown_connections 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5633314d7760 0x5633314d9b80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333145c240 0x563331460c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563331451980 0x563331460730 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 --2- 172.21.15.28:0/1951297268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563331451060 0x5633314601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f30f67fc640 0 client.16923 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.984+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 >> 172.21.15.28:0/1951297268 conn(0x56333144d5f0 msgr2=0x5633314598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.985+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 shutdown_connections 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.985+0000 7f31174cc740 1 -- 172.21.15.28:0/1951297268 wait complete. 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.985+0000 7f31174cc740 20 client.16923 trim_cache size 0 max 16384 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 39, 2024-09-19T09:40:46.988 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 32, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": { 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "c2823cd6": { 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 32, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "age_ref": 0.0, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "timeout": 90.0, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "expiration": 145.8, 2024-09-19T09:40:46.989 INFO:teuthology.orchestra.run.smithi028.stdout: "members": { 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: "file:/": { 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: "excluded": false, 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: "state": { 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RELEASED", 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:40:46.990 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.985+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 msgr2=0x7f30fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 0x7f30fc0a5700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f310800a2c0 tx=0x7f3108004a40 comp rx=0 tx=0).stop 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 msgr2=0x7f311019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009230 tx=0x7f30f8004270 comp rx=0 tx=0).stop 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 shutdown_connections 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f30fc0a3240 0x7f30fc0a5700 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3110106a80 0x7f311019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3110101420 0x7f311019abc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 --2- 172.21.15.28:0/1245234093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3110100a80 0x7f311019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:46.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 >> 172.21.15.28:0/1245234093 conn(0x7f31100fc7a0 msgr2=0x7f311006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:46.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 shutdown_connections 2024-09-19T09:40:46.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:40:46.986+0000 7f31150f3640 1 -- 172.21.15.28:0/1245234093 wait complete. 2024-09-19T09:40:47.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:47 smithi078 ceph-mon[35619]: pgmap v582: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T09:40:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:47 smithi038 ceph-mon[35271]: pgmap v582: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T09:40:47.716 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:47 smithi028 ceph-mon[30972]: pgmap v582: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T09:40:47.744 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c2823cd6', total seconds elapsed: 30.4 2024-09-19T09:40:49.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:49 smithi078 ceph-mon[35619]: pgmap v583: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T09:40:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:49 smithi038 ceph-mon[35271]: pgmap v583: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T09:40:49.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:49 smithi028 ceph-mon[30972]: pgmap v583: 97 pgs: 97 active+clean; 568 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T09:40:51.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:51 smithi078 ceph-mon[35619]: pgmap v584: 97 pgs: 97 active+clean; 566 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 35 op/s 2024-09-19T09:40:51.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:51 smithi038 ceph-mon[35271]: pgmap v584: 97 pgs: 97 active+clean; 566 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 35 op/s 2024-09-19T09:40:51.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:51 smithi028 ceph-mon[30972]: pgmap v584: 97 pgs: 97 active+clean; 566 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 35 op/s 2024-09-19T09:40:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:53 smithi078 ceph-mon[35619]: pgmap v585: 97 pgs: 97 active+clean; 563 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 164 KiB/s wr, 52 op/s 2024-09-19T09:40:53.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:53 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:53.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:53 smithi038 ceph-mon[35271]: pgmap v585: 97 pgs: 97 active+clean; 563 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 164 KiB/s wr, 52 op/s 2024-09-19T09:40:53.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:53 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:53.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:53 smithi028 ceph-mon[30972]: pgmap v585: 97 pgs: 97 active+clean; 563 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 164 KiB/s wr, 52 op/s 2024-09-19T09:40:53.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:53 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:54.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:54 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:54.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:54 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:54.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:54 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:55.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:55 smithi078 ceph-mon[35619]: pgmap v586: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 143 op/s 2024-09-19T09:40:55.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:55 smithi038 ceph-mon[35271]: pgmap v586: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 143 op/s 2024-09-19T09:40:55.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:55 smithi028 ceph-mon[30972]: pgmap v586: 97 pgs: 97 active+clean; 549 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 143 op/s 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/824711755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74107890 msgr2=0x7f8b74109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/824711755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74107890 0x7f8b74109ca0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009ca0 tx=0x7f8b6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/824711755 shutdown_connections 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/824711755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74104f60 0x7f8b74107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/824711755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74107890 0x7f8b74109ca0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/824711755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/824711755 >> 172.21.15.28:0/824711755 conn(0x7f8b740fc1b0 msgr2=0x7f8b740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/824711755 shutdown_connections 2024-09-19T09:40:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.492+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/824711755 wait complete. 2024-09-19T09:40:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 Processor -- start 2024-09-19T09:40:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 -- start start 2024-09-19T09:40:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74107890 0x7f8b7419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410d810 con 0x7f8b74104f60 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410d690 con 0x7f8b74107890 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b7a2c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7410d990 con 0x7f8b74102630 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52288/0 (socket says 172.21.15.28:52288) 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b737fe640 1 -- 172.21.15.28:0/1645675194 learned_addr learned my addr 172.21.15.28:0/1645675194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b73fff640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.493+0000 7f8b78866640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74107890 0x7f8b7419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 -- 172.21.15.28:0/1645675194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 msgr2=0x7f8b74197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 -- 172.21.15.28:0/1645675194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74107890 msgr2=0x7f8b7419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74107890 0x7f8b7419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 -- 172.21.15.28:0/1645675194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7419fb80 con 0x7f8b74104f60 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b73fff640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b737fe640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e9c0 tx=0x7f8b6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b717fa640 1 -- 172.21.15.28:0/1645675194 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800cd80 con 0x7f8b74104f60 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b717fa640 1 -- 172.21.15.28:0/1645675194 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68005500 con 0x7f8b74104f60 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7419fe70 con 0x7f8b74104f60 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b717fa640 1 -- 172.21.15.28:0/1645675194 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68010690 con 0x7f8b74104f60 2024-09-19T09:40:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.494+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a0330 con 0x7f8b74104f60 2024-09-19T09:40:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.497+0000 7f8b7d05e740 1 Processor -- start 2024-09-19T09:40:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.497+0000 7f8b717fa640 1 -- 172.21.15.28:0/1645675194 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8b680046c0 con 0x7f8b74104f60 2024-09-19T09:40:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.497+0000 7f8b7d05e740 1 -- start start 2024-09-19T09:40:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e5609ec2f0 con 0x55e560a8e370 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e560994270 con 0x55e560a892e0 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5609e82b0 con 0x55e560a889a0 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a90830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52294/0 (socket says 172.21.15.28:52294) 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b737fe640 1 -- 172.21.15.28:0/1965860419 learned_addr learned my addr 172.21.15.28:0/1965860419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b73fff640 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 -- 172.21.15.28:0/1965860419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 msgr2=0x55e560a8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 -- 172.21.15.28:0/1965860419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 msgr2=0x55e560a90830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b717fa640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 0x7f8b5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a90830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 -- 172.21.15.28:0/1965860419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e560993340 con 0x55e560a889a0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b717fa640 1 -- 172.21.15.28:0/1645675194 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b680cfc50 con 0x7f8b74104f60 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.498+0000 7f8b78866640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 0x7f8b5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.499+0000 7f8b78866640 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a88da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009bf0 tx=0x7f8b6c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b320186f33ef0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.499+0000 7f8b5a7fc640 1 -- 172.21.15.28:0/1965860419 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c031d30 con 0x55e560a889a0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.499+0000 7f8b5a7fc640 1 -- 172.21.15.28:0/1965860419 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c036040 con 0x55e560a889a0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.499+0000 7f8b5a7fc640 1 -- 172.21.15.28:0/1965860419 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c004250 con 0x55e560a889a0 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- 172.21.15.28:0/1965860419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 msgr2=0x55e560a88da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a88da0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009bf0 tx=0x7f8b6c004a40 comp rx=0 tx=0).stop 2024-09-19T09:40:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b78866640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 0x7f8b5c0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c046000 tx=0x7f8b6c04b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- 172.21.15.28:0/1965860419 shutdown_connections 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/1965860419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a88da0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- 172.21.15.28:0/1965860419 >> 172.21.15.28:0/1965860419 conn(0x55e560a84010 msgr2=0x55e560a84410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- 172.21.15.28:0/1965860419 shutdown_connections 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- 172.21.15.28:0/1965860419 wait complete. 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 -- start start 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 Processor -- start 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.500+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560a99f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e560a91160 con 0x55e560a892e0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e560975290 con 0x55e560a8e370 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e560a912e0 con 0x55e560a889a0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 10 client.0 _ll_register_callbacks cb 0x55e5609a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52308/0 (socket says 172.21.15.28:52308) 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 -- 172.21.15.28:0/2146884008 learned_addr learned my addr 172.21.15.28:0/2146884008 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b78866640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560a99f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 -- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 msgr2=0x55e560a929e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 -- 172.21.15.28:0/2146884008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 msgr2=0x55e560a99f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560a99f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 -- 172.21.15.28:0/2146884008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5606ff210 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000c00 tx=0x7f8b680d3320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f30f4a6b1589f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6809bde0 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b680a4c80 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6c036040 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.501+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8b6809bc00 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 10 client.16935 fetch_fsmap learned FSMap version 42 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e5607cb620 con 0x55e560a892e0 2024-09-19T09:40:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c036040 con 0x55e560a892e0 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f8b680bc0c0 con 0x55e560a892e0 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 10 client.16935 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 20 client.16935 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560aa0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 4 client.16935 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5609f9a20 con 0x55e560a9e660 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.502+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560aa0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.503+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560aa0a50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47044056b4282bf2 server_cookie=22e6f7344743b821 in_seq=0 out_seq=0 2024-09-19T09:40:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.503+0000 7f8b537fe640 10 client.16935 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b537fe640 1 -- 172.21.15.28:0/2146884008 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5609f9a20 con 0x55e560a9e660 2024-09-19T09:40:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b537fe640 10 client.16935 handle_command_reply: tid=0 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 client.16935 shutdown 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 msgr2=0x55e560a92f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000c00 tx=0x7f8b680d3320 comp rx=0 tx=0).stop 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 shutdown_connections 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560a99f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560aa0a50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b537fe640 0 client.16935 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a892e0 0x55e560a92f20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2146884008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 >> 172.21.15.28:0/2146884008 conn(0x55e560a84010 msgr2=0x55e560a85820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 shutdown_connections 2024-09-19T09:40:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2146884008 wait complete. 2024-09-19T09:40:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.504+0000 7f8b7d05e740 20 client.16935 trim_cache size 0 max 16384 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.522+0000 7f8b7d05e740 1 Processor -- start 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.522+0000 7f8b7d05e740 1 -- start start 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.522+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a99f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741a90f0 con 0x55e560a8e370 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741aacc0 con 0x55e560a892e0 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741a97f0 con 0x55e560a889a0 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52326/0 (socket says 172.21.15.28:52326) 2024-09-19T09:40:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 -- 172.21.15.28:0/528548253 learned_addr learned my addr 172.21.15.28:0/528548253 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b737fe640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a99f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b73fff640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 -- 172.21.15.28:0/528548253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 msgr2=0x55e560a929e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 -- 172.21.15.28:0/528548253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 msgr2=0x55e560a99f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a99f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 -- 172.21.15.28:0/528548253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e560a897c0 con 0x55e560a8e370 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b737fe640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a99f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b78866640 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c003ed0 tx=0x7f8b6c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e02bed0147140cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b59ffb640 1 -- 172.21.15.28:0/528548253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c053070 con 0x55e560a8e370 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b59ffb640 1 -- 172.21.15.28:0/528548253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c054040 con 0x55e560a8e370 2024-09-19T09:40:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.523+0000 7f8b59ffb640 1 -- 172.21.15.28:0/528548253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c004da0 con 0x55e560a8e370 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 -- 172.21.15.28:0/528548253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 msgr2=0x55e560a92f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c003ed0 tx=0x7f8b6c0319a0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 -- 172.21.15.28:0/528548253 shutdown_connections 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a892e0 0x55e560a99f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a8e370 0x55e560a92f20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/528548253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a889a0 0x55e560a929e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 -- 172.21.15.28:0/528548253 >> 172.21.15.28:0/528548253 conn(0x55e560a84010 msgr2=0x55e560a8db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 -- 172.21.15.28:0/528548253 shutdown_connections 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.524+0000 7f8b7d05e740 1 -- 172.21.15.28:0/528548253 wait complete. 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 -- start start 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 Processor -- start 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 0x55e560a97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560b07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741aa5c0 con 0x55e560a889a0 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741a9ec0 con 0x55e560a8e370 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741a70d0 con 0x55e560a892e0 2024-09-19T09:40:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b7d05e740 10 client.0 _ll_register_callbacks cb 0x7f8b680d3420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b73fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b73fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52338/0 (socket says 172.21.15.28:52338) 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 0x55e560a97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b73fff640 1 -- 172.21.15.28:0/2680234861 learned_addr learned my addr 172.21.15.28:0/2680234861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b78866640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560b07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.525+0000 7f8b73fff640 1 -- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 msgr2=0x55e560a97450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 0x55e560a97450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b73fff640 1 -- 172.21.15.28:0/2680234861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 msgr2=0x55e560b07320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560b07320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b73fff640 1 -- 172.21.15.28:0/2680234861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e560b07860 con 0x55e560a889a0 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 0x55e560a97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:40:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b73fff640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55e560a84e90 tx=0x7f8b6401f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2c5e2d2305e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640120d0 con 0x55e560a889a0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f8b64012270 con 0x55e560a889a0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6c054040 con 0x55e560a889a0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8b640035f0 con 0x55e560a889a0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 10 client.16941 fetch_fsmap learned FSMap version 42 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e560b08c40 con 0x55e560a889a0 2024-09-19T09:40:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c054040 con 0x55e560a889a0 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f8b640107b0 con 0x55e560a889a0 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 10 client.16941 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 20 client.16941 populate_metadata read hostname 'smithi028' 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560b0b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.526+0000 7f8b7d05e740 4 client.16941 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.527+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5609f3d70 con 0x55e560a9e660 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.527+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560b0b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.527+0000 7f8b537fe640 10 client.16941 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.527+0000 7f8b737fe640 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560b0b6b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c7d51317cd0bfb server_cookie=f52176a47b6646a1 in_seq=0 out_seq=0 2024-09-19T09:40:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.755+0000 7f8b537fe640 1 -- 172.21.15.28:0/2680234861 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5609f3d70 con 0x55e560a9e660 2024-09-19T09:40:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.755+0000 7f8b537fe640 10 client.16941 handle_command_reply: tid=0 2024-09-19T09:40:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.755+0000 7f8b7d05e740 1 client.16941 shutdown 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.755+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 msgr2=0x55e560a96f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.755+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55e560a84e90 tx=0x7f8b6401f7c0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 shutdown_connections 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e560a8e370 0x55e560b07320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55e560a9e660 0x55e560b0b6b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e560a892e0 0x55e560a97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 --2- 172.21.15.28:0/2680234861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e560a889a0 0x55e560a96f10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 >> 172.21.15.28:0/2680234861 conn(0x55e560a84010 msgr2=0x55e560a8a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b537fe640 0 client.16941 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 shutdown_connections 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 1 -- 172.21.15.28:0/2680234861 wait complete. 2024-09-19T09:40:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7d05e740 20 client.16941 trim_cache size 0 max 16384 2024-09-19T09:40:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:40:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:40:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:40:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 msgr2=0x7f8b5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.756+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 0x7f8b5c0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c046000 tx=0x7f8b6c04b000 comp rx=0 tx=0).stop 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 msgr2=0x7f8b741984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e9c0 tx=0x7f8b6800ee80 comp rx=0 tx=0).stop 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 shutdown_connections 2024-09-19T09:40:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f8b5c0a3240 0x7f8b5c0a5700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b74107890 0x7f8b7419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b74104f60 0x7f8b741984a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 --2- 172.21.15.28:0/1645675194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b74102630 0x7f8b74197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 >> 172.21.15.28:0/1645675194 conn(0x7f8b740fc1b0 msgr2=0x7f8b74102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 shutdown_connections 2024-09-19T09:40:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:40:56.757+0000 7f8b7a2c9640 1 -- 172.21.15.28:0/1645675194 wait complete. 2024-09-19T09:40:57.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-mon[30972]: pgmap v587: 97 pgs: 97 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 768 KiB/s wr, 195 op/s 2024-09-19T09:40:57.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-mon[30972]: daemon mds.d restarted 2024-09-19T09:40:57.514 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ignoring --setuser ceph since I am not root 2024-09-19T09:40:57.514 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ignoring --setgroup ceph since I am not root 2024-09-19T09:40:57.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-mds[96943]: -- 172.21.15.28:0/1155443004 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609b6d1bc20 con 0x5609b6cf8800 2024-09-19T09:40:57.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:40:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: starting mds.d at 2024-09-19T09:40:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:57 smithi078 ceph-mon[35619]: pgmap v587: 97 pgs: 97 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 768 KiB/s wr, 195 op/s 2024-09-19T09:40:57.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:57 smithi078 ceph-mon[35619]: daemon mds.d restarted 2024-09-19T09:40:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:57 smithi038 ceph-mon[35271]: pgmap v587: 97 pgs: 97 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 768 KiB/s wr, 195 op/s 2024-09-19T09:40:57.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:57 smithi038 ceph-mon[35271]: daemon mds.d restarted 2024-09-19T09:40:58.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:58 smithi078 ceph-mon[35619]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:40:58.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:58 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6836/3826354443,v1:172.21.15.28:6837/3826354443] up:boot 2024-09-19T09:40:58.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:58 smithi078 ceph-mon[35619]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:40:58.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:58 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:40:58.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:58 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:58.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:58 smithi038 ceph-mon[35271]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:40:58.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:58 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6836/3826354443,v1:172.21.15.28:6837/3826354443] up:boot 2024-09-19T09:40:58.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:58 smithi038 ceph-mon[35271]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:40:58.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:58 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:40:58.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:58 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:58 smithi028 ceph-mon[30972]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:40:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:58 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6836/3826354443,v1:172.21.15.28:6837/3826354443] up:boot 2024-09-19T09:40:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:58 smithi028 ceph-mon[30972]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:40:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:58 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:40:58.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:58 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:40:59.665 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:40:59 smithi028 ceph-mon[30972]: pgmap v589: 97 pgs: 97 active+clean; 535 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 939 KiB/s wr, 288 op/s 2024-09-19T09:40:59.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:40:59 smithi038 ceph-mon[35271]: pgmap v589: 97 pgs: 97 active+clean; 535 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 939 KiB/s wr, 288 op/s 2024-09-19T09:40:59.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:40:59 smithi078 ceph-mon[35619]: pgmap v589: 97 pgs: 97 active+clean; 535 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 939 KiB/s wr, 288 op/s 2024-09-19T09:41:01.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:01 smithi038 ceph-mon[35271]: pgmap v590: 97 pgs: 97 active+clean; 501 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-09-19T09:41:01.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:01 smithi028 ceph-mon[30972]: pgmap v590: 97 pgs: 97 active+clean; 501 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-09-19T09:41:01.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:01 smithi078 ceph-mon[35619]: pgmap v590: 97 pgs: 97 active+clean; 501 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-09-19T09:41:03.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:03 smithi038 ceph-mon[35271]: pgmap v591: 97 pgs: 97 active+clean; 500 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 480 op/s 2024-09-19T09:41:03.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:03 smithi028 ceph-mon[30972]: pgmap v591: 97 pgs: 97 active+clean; 500 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 480 op/s 2024-09-19T09:41:03.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:03 smithi078 ceph-mon[35619]: pgmap v591: 97 pgs: 97 active+clean; 500 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 480 op/s 2024-09-19T09:41:05.659 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:05 smithi078 ceph-mon[35619]: pgmap v592: 97 pgs: 97 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 503 op/s 2024-09-19T09:41:05.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:05 smithi038 ceph-mon[35271]: pgmap v592: 97 pgs: 97 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 503 op/s 2024-09-19T09:41:05.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:05 smithi028 ceph-mon[30972]: pgmap v592: 97 pgs: 97 active+clean; 482 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 503 op/s 2024-09-19T09:41:07.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:07 smithi038 ceph-mon[35271]: pgmap v593: 97 pgs: 97 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 501 op/s 2024-09-19T09:41:07.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:07 smithi028 ceph-mon[30972]: pgmap v593: 97 pgs: 97 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 501 op/s 2024-09-19T09:41:07.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:07 smithi078 ceph-mon[35619]: pgmap v593: 97 pgs: 97 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 501 op/s 2024-09-19T09:41:08.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:08 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:08.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:08 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:08.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:08 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:09.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:09 smithi038 ceph-mon[35271]: pgmap v594: 97 pgs: 97 active+clean; 467 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 479 op/s 2024-09-19T09:41:09.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:09 smithi028 ceph-mon[30972]: pgmap v594: 97 pgs: 97 active+clean; 467 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 479 op/s 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- 172.21.15.28:0/1741203863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528101b40 msgr2=0x7f0528101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- 172.21.15.28:0/1741203863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528101b40 0x7f0528101f40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0518009900 tx=0x7f051802a410 comp rx=0 tx=0).stop 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- 172.21.15.28:0/1741203863 shutdown_connections 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- 172.21.15.28:0/1741203863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05281036f0 0x7f0528109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- 172.21.15.28:0/1741203863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0528102d30 0x7f05281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- 172.21.15.28:0/1741203863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528101b40 0x7f0528101f40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- 172.21.15.28:0/1741203863 >> 172.21.15.28:0/1741203863 conn(0x7f05280fd2e0 msgr2=0x7f05280ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- 172.21.15.28:0/1741203863 shutdown_connections 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- 172.21.15.28:0/1741203863 wait complete. 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 Processor -- start 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 -- start start 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0528101b40 0x7f0528197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.849+0000 7f052e45e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05281036f0 0x7f052819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f052810d7a0 con 0x7f0528102d30 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f052810d620 con 0x7f0528101b40 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f052810d920 con 0x7f05281036f0 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59418/0 (socket says 172.21.15.28:59418) 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 -- 172.21.15.28:0/806851621 learned_addr learned my addr 172.21.15.28:0/806851621 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052c9fb640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05281036f0 0x7f052819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f0527fff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0528101b40 0x7f0528197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 -- 172.21.15.28:0/806851621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05281036f0 msgr2=0x7f052819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05281036f0 0x7f052819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 -- 172.21.15.28:0/806851621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0528101b40 msgr2=0x7f0528197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0528101b40 0x7f0528197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 -- 172.21.15.28:0/806851621 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819faa0 con 0x7f0528102d30 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f051ffff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f051000e9f0 tx=0x7f051000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f0525ffb640 1 -- 172.21.15.28:0/806851621 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051000cd80 con 0x7f0528102d30 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f0525ffb640 1 -- 172.21.15.28:0/806851621 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0510010910 con 0x7f0528102d30 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819fd90 con 0x7f0528102d30 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f0525ffb640 1 -- 172.21.15.28:0/806851621 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510018990 con 0x7f0528102d30 2024-09-19T09:41:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.850+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a0260 con 0x7f0528102d30 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.853+0000 7f05311f7740 1 Processor -- start 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.853+0000 7f05311f7740 1 -- start start 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f0525ffb640 1 -- 172.21.15.28:0/806851621 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0510010ac0 con 0x7f0528102d30 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fbb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5579b2f172f0 con 0x5579b2fb9370 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5579b2ebf270 con 0x5579b2fb42e0 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f05311f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5579b2f132b0 con 0x5579b2fb39a0 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f051ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fbb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:38966/0 (socket says 172.21.15.28:38966) 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 -- 172.21.15.28:0/445460782 learned_addr learned my addr 172.21.15.28:0/445460782 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f0527fff640 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 -- 172.21.15.28:0/445460782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 msgr2=0x5579b2fb8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fb8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 -- 172.21.15.28:0/445460782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 msgr2=0x5579b2fbb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fbb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 -- 172.21.15.28:0/445460782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b2ebe340 con 0x5579b2fb39a0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.854+0000 7f052c9fb640 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0520010370 tx=0x7f0520010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb7aa9533644d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f0525ffb640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 0x7f05140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f051e7fc640 1 -- 172.21.15.28:0/445460782 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520016070 con 0x5579b2fb39a0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f051e7fc640 1 -- 172.21.15.28:0/445460782 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520004d20 con 0x5579b2fb39a0 2024-09-19T09:41:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f051e7fc640 1 -- 172.21.15.28:0/445460782 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200053e0 con 0x5579b2fb39a0 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f0527fff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 0x7f05140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f0525ffb640 1 -- 172.21.15.28:0/806851621 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f05100cfb50 con 0x7f0528102d30 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 -- 172.21.15.28:0/445460782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 msgr2=0x5579b2fb3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f0527fff640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 0x7f05140a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0518005ab0 tx=0x7f0518005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0520010370 tx=0x7f0520010830 comp rx=0 tx=0).stop 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 -- 172.21.15.28:0/445460782 shutdown_connections 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fb8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fbb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 --2- 172.21.15.28:0/445460782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fb3da0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 -- 172.21.15.28:0/445460782 >> 172.21.15.28:0/445460782 conn(0x5579b2faf010 msgr2=0x5579b2faf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.855+0000 7f05311f7740 1 -- 172.21.15.28:0/445460782 shutdown_connections 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 -- 172.21.15.28:0/445460782 wait complete. 2024-09-19T09:41:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 -- start start 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 Processor -- start 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb39a0 0x5579b2fbdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5579b2fbc340 con 0x5579b2fb42e0 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5579b2ea0290 con 0x5579b2fb39a0 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5579b2fbc4c0 con 0x5579b2fb9370 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f05311f7740 10 client.0 _ll_register_callbacks cb 0x5579b2ed1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f051ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f051ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59428/0 (socket says 172.21.15.28:59428) 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:38968/0 (socket says 172.21.15.28:38968) 2024-09-19T09:41:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f051ffff640 1 -- 172.21.15.28:0/1436317210 learned_addr learned my addr 172.21.15.28:0/1436317210 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.856+0000 7f0527fff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb39a0 0x5579b2fbdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 -- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 msgr2=0x5579b2fc5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 -- 172.21.15.28:0/1436317210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb39a0 msgr2=0x5579b2fbdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb39a0 0x5579b2fbdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 -- 172.21.15.28:0/1436317210 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b2c2a210 con 0x5579b2fb42e0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f052c9fb640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f051ffff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0510098c20 tx=0x7f05100985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8841bfa60815af4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05077fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051009c250 con 0x5579b2fb42e0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f05100bc0e0 con 0x5579b2fb42e0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0520004d20 con 0x5579b2fb42e0 2024-09-19T09:41:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f05100a0550 con 0x5579b2fb42e0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05311f7740 10 client.16959 fetch_fsmap learned FSMap version 45 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579b2cf6620 con 0x5579b2fb42e0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.857+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520004d20 con 0x5579b2fb42e0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f05100cdcd0 con 0x5579b2fb42e0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 10 client.16959 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 20 client.16959 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b2fc9840 0x5579b2fcbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 4 client.16959 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b2f24a20 con 0x5579b2fc9840 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f0527fff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b2fc9840 0x5579b2fcbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f0527fff640 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b2fc9840 0x5579b2fcbc30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41d1bb70d98c0a21 server_cookie=11d6a7f6800004b4 in_seq=0 out_seq=0 2024-09-19T09:41:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.858+0000 7f05077fe640 10 client.16959 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:09.867 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:09 smithi078 ceph-mon[35619]: pgmap v594: 97 pgs: 97 active+clean; 467 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 479 op/s 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05077fe640 1 -- 172.21.15.28:0/1436317210 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579b2f24a20 con 0x5579b2fc9840 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05077fe640 10 client.16959 handle_command_reply: tid=0 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 client.16959 shutdown 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 msgr2=0x5579b2fbe110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0510098c20 tx=0x7f05100985b0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 shutdown_connections 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b2fc9840 0x5579b2fcbc30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb42e0 0x5579b2fbe110 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 --2- 172.21.15.28:0/1436317210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb39a0 0x5579b2fbdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05077fe640 0 client.16959 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 >> 172.21.15.28:0/1436317210 conn(0x5579b2faf010 msgr2=0x5579b2fb0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 shutdown_connections 2024-09-19T09:41:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 1 -- 172.21.15.28:0/1436317210 wait complete. 2024-09-19T09:41:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.860+0000 7f05311f7740 20 client.16959 trim_cache size 0 max 16384 2024-09-19T09:41:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 Processor -- start 2024-09-19T09:41:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 -- start start 2024-09-19T09:41:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fb0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a9050 con 0x5579b2fb9370 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05281aac80 con 0x5579b2fb42e0 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f05311f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a97b0 con 0x5579b2fb39a0 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fb0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:38972/0 (socket says 172.21.15.28:38972) 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 -- 172.21.15.28:0/2081996646 learned_addr learned my addr 172.21.15.28:0/2081996646 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f051ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f052c9fb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fb0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59440/0 (socket says 172.21.15.28:59440) 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 -- 172.21.15.28:0/2081996646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 msgr2=0x5579b2fc5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 -- 172.21.15.28:0/2081996646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 msgr2=0x5579b2fb0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fb0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.875+0000 7f0527fff640 1 -- 172.21.15.28:0/2081996646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b2fc65f0 con 0x5579b2fb39a0 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f0527fff640 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0518005c40 tx=0x7f051803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9259c87a4050f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f051dffb640 1 -- 172.21.15.28:0/2081996646 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518057070 con 0x5579b2fb39a0 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f051dffb640 1 -- 172.21.15.28:0/2081996646 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0518005480 con 0x5579b2fb39a0 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f051dffb640 1 -- 172.21.15.28:0/2081996646 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518035040 con 0x5579b2fb39a0 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 -- 172.21.15.28:0/2081996646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 msgr2=0x5579b2fae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0518005c40 tx=0x7f051803f000 comp rx=0 tx=0).stop 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 -- 172.21.15.28:0/2081996646 shutdown_connections 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb9370 0x5579b2fb0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 --2- 172.21.15.28:0/2081996646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb39a0 0x5579b2fae0b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 -- 172.21.15.28:0/2081996646 >> 172.21.15.28:0/2081996646 conn(0x5579b2faf010 msgr2=0x5579b2fad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 -- 172.21.15.28:0/2081996646 shutdown_connections 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.876+0000 7f05311f7740 1 -- 172.21.15.28:0/2081996646 wait complete. 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 -- start start 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 Processor -- start 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05281aa5b0 con 0x5579b2fb39a0 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a9eb0 con 0x5579b2fb42e0 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a7030 con 0x5579b2fb9370 2024-09-19T09:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f05311f7740 10 client.0 _ll_register_callbacks cb 0x7f05100d3320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f0527fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f0527fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59446/0 (socket says 172.21.15.28:59446) 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f0527fff640 1 -- 172.21.15.28:0/4195657593 learned_addr learned my addr 172.21.15.28:0/4195657593 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f052c9fb640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.877+0000 7f051ffff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 -- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 msgr2=0x5579b2fc3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 -- 172.21.15.28:0/4195657593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 msgr2=0x5579b2fc2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 -- 172.21.15.28:0/4195657593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b2fc4400 con 0x5579b2fb39a0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f0527fff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0518009560 tx=0x7f0518009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457e64b0441078c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518057040 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f0518004780 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0518005480 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0518005630 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05311f7740 10 client.16965 fetch_fsmap learned FSMap version 45 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579b30344f0 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0518005480 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f0518067480 con 0x5579b2fb39a0 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.878+0000 7f05311f7740 10 client.16965 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05311f7740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05311f7740 20 client.16965 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b3034f00 0x5579b3037320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05311f7740 4 client.16965 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:41:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579b2f1ed70 con 0x5579b3034f00 2024-09-19T09:41:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f051ffff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b3034f00 0x5579b3037320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f05077fe640 10 client.16965 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:09.879+0000 7f051ffff640 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b3034f00 0x5579b3037320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d8d3c2cf0abaed server_cookie=16d9eb529e699670 in_seq=0 out_seq=0 2024-09-19T09:41:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05077fe640 1 -- 172.21.15.28:0/4195657593 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579b2f1ed70 con 0x5579b3034f00 2024-09-19T09:41:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05077fe640 10 client.16965 handle_command_reply: tid=0 2024-09-19T09:41:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 client.16965 shutdown 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 msgr2=0x5579b2fc28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0518009560 tx=0x7f0518009590 comp rx=0 tx=0).stop 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 shutdown_connections 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579b2fb42e0 0x5579b2fc2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x5579b3034f00 0x5579b3037320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5579b2fb9370 0x5579b2fc3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 --2- 172.21.15.28:0/4195657593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5579b2fb39a0 0x5579b2fc28a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05077fe640 0 client.16965 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 >> 172.21.15.28:0/4195657593 conn(0x5579b2faf010 msgr2=0x5579b2fae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 shutdown_connections 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 1 -- 172.21.15.28:0/4195657593 wait complete. 2024-09-19T09:41:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.219+0000 7f05311f7740 20 client.16965 trim_cache size 0 max 16384 2024-09-19T09:41:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:41:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:41:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:41:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:41:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 msgr2=0x7f05140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 0x7f05140a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0518005ab0 tx=0x7f0518005fd0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 msgr2=0x7f05281983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f051000e9f0 tx=0x7f051000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 shutdown_connections 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05140a32c0 0x7f05140a5780 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05281036f0 0x7f052819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0528102d30 0x7f05281983c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 --2- 172.21.15.28:0/806851621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0528101b40 0x7f0528197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 >> 172.21.15.28:0/806851621 conn(0x7f05280fd2e0 msgr2=0x7f0528106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 shutdown_connections 2024-09-19T09:41:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:10.220+0000 7f052e45e640 1 -- 172.21.15.28:0/806851621 wait complete. 2024-09-19T09:41:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:11 smithi038 ceph-mon[35271]: pgmap v595: 97 pgs: 97 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 524 op/s 2024-09-19T09:41:11.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:11 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:41:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:11 smithi028 ceph-mon[30972]: pgmap v595: 97 pgs: 97 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 524 op/s 2024-09-19T09:41:11.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:11 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:41:11.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:11 smithi078 ceph-mon[35619]: pgmap v595: 97 pgs: 97 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 524 op/s 2024-09-19T09:41:11.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:11 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:41:12.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:41:12.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:41:12.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:41:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:41:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:41:12.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:41:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:41:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:41:12.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:41:13.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:13 smithi038 ceph-mon[35271]: pgmap v596: 97 pgs: 97 active+clean; 439 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 396 op/s 2024-09-19T09:41:13.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:13 smithi028 ceph-mon[30972]: pgmap v596: 97 pgs: 97 active+clean; 439 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 396 op/s 2024-09-19T09:41:13.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:13 smithi078 ceph-mon[35619]: pgmap v596: 97 pgs: 97 active+clean; 439 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 396 op/s 2024-09-19T09:41:14.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:14 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:14.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:14 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:14.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:14 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:15.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:15 smithi038 ceph-mon[35271]: pgmap v597: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-09-19T09:41:15.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:15 smithi028 ceph-mon[30972]: pgmap v597: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-09-19T09:41:15.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:15 smithi078 ceph-mon[35619]: pgmap v597: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-09-19T09:41:17.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:17 smithi028 ceph-mon[30972]: pgmap v598: 97 pgs: 97 active+clean; 406 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-19T09:41:17.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:17 smithi078 ceph-mon[35619]: pgmap v598: 97 pgs: 97 active+clean; 406 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-19T09:41:17.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:17 smithi038 ceph-mon[35271]: pgmap v598: 97 pgs: 97 active+clean; 406 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-19T09:41:19.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:19 smithi028 ceph-mon[30972]: pgmap v599: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-09-19T09:41:19.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:19 smithi078 ceph-mon[35619]: pgmap v599: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-09-19T09:41:19.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:19 smithi038 ceph-mon[35271]: pgmap v599: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-09-19T09:41:21.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:21 smithi028 ceph-mon[30972]: pgmap v600: 97 pgs: 97 active+clean; 378 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-09-19T09:41:21.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:21 smithi078 ceph-mon[35619]: pgmap v600: 97 pgs: 97 active+clean; 378 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-09-19T09:41:21.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:21 smithi038 ceph-mon[35271]: pgmap v600: 97 pgs: 97 active+clean; 378 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-09-19T09:41:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 -- 172.21.15.28:0/1364249310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 msgr2=0x7fdd90108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/1364249310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd90108e80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009ca0 tx=0x7fdd8802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 -- 172.21.15.28:0/1364249310 shutdown_connections 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/1364249310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 0x7fdd901020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/1364249310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd90108e80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/1364249310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 0x7fdd90100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 -- 172.21.15.28:0/1364249310 >> 172.21.15.28:0/1364249310 conn(0x7fdd900fc1c0 msgr2=0x7fdd900fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 -- 172.21.15.28:0/1364249310 shutdown_connections 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.310+0000 7fdd95f1a640 1 -- 172.21.15.28:0/1364249310 wait complete. 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 Processor -- start 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 -- start start 2024-09-19T09:41:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 0x7fdd9006a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 0x7fdd9019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010c710 con 0x7fdd901025f0 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010c590 con 0x7fdd90101c30 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd95f1a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010c890 con 0x7fdd90100a40 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 0x7fdd9006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49002/0 (socket says 172.21.15.28:49002) 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 -- 172.21.15.28:0/3466108577 learned_addr learned my addr 172.21.15.28:0/3466108577 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8effd640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 0x7fdd9019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 -- 172.21.15.28:0/3466108577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 msgr2=0x7fdd9006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 0x7fdd9006a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 -- 172.21.15.28:0/3466108577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 msgr2=0x7fdd9019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 0x7fdd9019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.311+0000 7fdd8ffff640 1 -- 172.21.15.28:0/3466108577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9019fa90 con 0x7fdd901025f0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd8ffff640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdd8802f540 tx=0x7fdd88031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd8cff9640 1 -- 172.21.15.28:0/3466108577 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88003b70 con 0x7fdd901025f0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9019fcc0 con 0x7fdd901025f0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd8cff9640 1 -- 172.21.15.28:0/3466108577 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd88003d10 con 0x7fdd901025f0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd8cff9640 1 -- 172.21.15.28:0/3466108577 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8802fb50 con 0x7fdd901025f0 2024-09-19T09:41:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.312+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a01c0 con 0x7fdd901025f0 2024-09-19T09:41:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.314+0000 7fdd8cff9640 1 -- 172.21.15.28:0/3466108577 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fdd8802fcf0 con 0x7fdd901025f0 2024-09-19T09:41:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 Processor -- start 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 -- start start 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b269da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b271830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a8b1cd2f0 con 0x555a8b26f370 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a8b175270 con 0x555a8b2699a0 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd98caf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a8b1c92b0 con 0x555a8b26a2e0 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b271830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:59924/0 (socket says 172.21.15.28:59924) 2024-09-19T09:41:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b271830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49016/0 (socket says 172.21.15.28:49016) 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 -- 172.21.15.28:0/956016827 learned_addr learned my addr 172.21.15.28:0/956016827 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8cff9640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 0x7fdd780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8effd640 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b269da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 -- 172.21.15.28:0/956016827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 msgr2=0x555a8b269da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b269da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 -- 172.21.15.28:0/956016827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 msgr2=0x555a8b271830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b271830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8f7fe640 1 -- 172.21.15.28:0/956016827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8b174340 con 0x555a8b26a2e0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.315+0000 7fdd8ffff640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 0x7fdd780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd8cff9640 1 -- 172.21.15.28:0/3466108577 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fdd880f0ce0 con 0x7fdd901025f0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009ca0 tx=0x7fdd80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31525dbb67c88c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd75ffb640 1 -- 172.21.15.28:0/956016827 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004220 con 0x555a8b26a2e0 2024-09-19T09:41:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd75ffb640 1 -- 172.21.15.28:0/956016827 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd800043c0 con 0x555a8b26a2e0 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd75ffb640 1 -- 172.21.15.28:0/956016827 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80019900 con 0x555a8b26a2e0 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.316+0000 7fdd8ffff640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 0x7fdd780a5700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdd880bd260 tx=0x7fdd880882d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- 172.21.15.28:0/956016827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 msgr2=0x555a8b26eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009ca0 tx=0x7fdd80011810 comp rx=0 tx=0).stop 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- 172.21.15.28:0/956016827 shutdown_connections 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b271830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b26eda0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- 172.21.15.28:0/956016827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b269da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- 172.21.15.28:0/956016827 >> 172.21.15.28:0/956016827 conn(0x555a8b265010 msgr2=0x555a8b265410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- 172.21.15.28:0/956016827 shutdown_connections 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- 172.21.15.28:0/956016827 wait complete. 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- start start 2024-09-19T09:41:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 Processor -- start 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b273bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b274130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a8b272340 con 0x555a8b26f370 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a8b156290 con 0x555a8b2699a0 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.317+0000 7fdd98caf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a8b2724c0 con 0x555a8b26a2e0 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd98caf740 10 client.0 _ll_register_callbacks cb 0x555a8b187140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b274130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49018/0 (socket says 172.21.15.28:49018) 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 -- 172.21.15.28:0/23050735 learned_addr learned my addr 172.21.15.28:0/23050735 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b273bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 -- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 msgr2=0x555a8b274130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b274130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 -- 172.21.15.28:0/23050735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 msgr2=0x555a8b273bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b273bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 -- 172.21.15.28:0/23050735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8aee0210 con 0x555a8b26f370 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd8effd640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400f410 tx=0x7fdd84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423d645c901ba2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e5c0 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fdd8400e760 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd800043c0 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdd84029070 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd98caf740 10 client.16977 fetch_fsmap learned FSMap version 46 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.318+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555a8afac620 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd800043c0 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 46) ==== 8390+0+0 (secure 0 0 0) 0x7fdd84020040 con 0x555a8b26f370 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 10 client.16977 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 20 client.16977 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b27f7d0 0x555a8b281c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 4 client.16977 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8b1daa20 con 0x555a8b27f7d0 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b27f7d0 0x555a8b281c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd6effd640 10 client.16977 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.319+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b27f7d0 0x555a8b281c40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e97b665fa3359b4 server_cookie=726d7840182f401e in_seq=0 out_seq=0 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd6effd640 1 -- 172.21.15.28:0/23050735 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a8b1daa20 con 0x555a8b27f7d0 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd6effd640 10 client.16977 handle_command_reply: tid=0 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 client.16977 shutdown 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 msgr2=0x555a8b27b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400f410 tx=0x7fdd84004730 comp rx=0 tx=0).stop 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 shutdown_connections 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b27f7d0 0x555a8b281c40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b27b100 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b274130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd6effd640 0 client.16977 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 --2- 172.21.15.28:0/23050735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b273bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 >> 172.21.15.28:0/23050735 conn(0x555a8b265010 msgr2=0x555a8b266820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 shutdown_connections 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 1 -- 172.21.15.28:0/23050735 wait complete. 2024-09-19T09:41:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.321+0000 7fdd98caf740 20 client.16977 trim_cache size 0 max 16384 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 Processor -- start 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 -- start start 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 0x555a8b265ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b26f370 0x555a8b266400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901a8ee0 con 0x555a8b26a2e0 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aab40 con 0x555a8b26f370 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd98caf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901a9640 con 0x555a8b2699a0 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 0x555a8b265ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49024/0 (socket says 172.21.15.28:49024) 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8f7fe640 1 -- 172.21.15.28:0/3245042498 learned_addr learned my addr 172.21.15.28:0/3245042498 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 -- 172.21.15.28:0/3245042498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b26f370 msgr2=0x555a8b266400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b26f370 0x555a8b266400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 -- 172.21.15.28:0/3245042498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 msgr2=0x555a8b27b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 -- 172.21.15.28:0/3245042498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8b2822c0 con 0x555a8b2699a0 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.336+0000 7fdd8effd640 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 0x555a8b265ec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097c0 tx=0x7fdd84005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f12eaf429db294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd757fa640 1 -- 172.21.15.28:0/3245042498 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84022460 con 0x555a8b2699a0 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd757fa640 1 -- 172.21.15.28:0/3245042498 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd84032020 con 0x555a8b2699a0 2024-09-19T09:41:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd757fa640 1 -- 172.21.15.28:0/3245042498 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84022600 con 0x555a8b2699a0 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 -- 172.21.15.28:0/3245042498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 msgr2=0x555a8b265ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 0x555a8b265ec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097c0 tx=0x7fdd84005f90 comp rx=0 tx=0).stop 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 -- 172.21.15.28:0/3245042498 shutdown_connections 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b26f370 0x555a8b266400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26a2e0 0x555a8b27b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 --2- 172.21.15.28:0/3245042498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b2699a0 0x555a8b265ec0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 -- 172.21.15.28:0/3245042498 >> 172.21.15.28:0/3245042498 conn(0x555a8b265010 msgr2=0x555a8b26b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 -- 172.21.15.28:0/3245042498 shutdown_connections 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.337+0000 7fdd98caf740 1 -- 172.21.15.28:0/3245042498 wait complete. 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 -- start start 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 Processor -- start 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b278570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b278ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aa470 con 0x555a8b26f370 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901a9d70 con 0x555a8b2699a0 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901a6ec0 con 0x555a8b26a2e0 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd98caf740 10 client.0 _ll_register_callbacks cb 0x7fdd84009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49040/0 (socket says 172.21.15.28:49040) 2024-09-19T09:41:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd8effd640 1 -- 172.21.15.28:0/1940864875 learned_addr learned my addr 172.21.15.28:0/1940864875 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b278ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.338+0000 7fdd8effd640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b278570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 -- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 msgr2=0x555a8b278ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b278ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 -- 172.21.15.28:0/1940864875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 msgr2=0x555a8b278570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b278570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 -- 172.21.15.28:0/1940864875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8b27a0d0 con 0x555a8b26f370 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b278ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd8effd640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdd84022880 tx=0x7fdd84009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617600728e5864d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84022950 con 0x555a8b26f370 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fdd8401a070 con 0x555a8b26f370 2024-09-19T09:41:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd84032020 con 0x555a8b26f370 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdd84041050 con 0x555a8b26f370 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd98caf740 10 client.16983 fetch_fsmap learned FSMap version 46 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555a8b27b3e0 con 0x555a8b26f370 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd84032020 con 0x555a8b26f370 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.339+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 46) ==== 8390+0+0 (secure 0 0 0) 0x7fdd84039040 con 0x555a8b26f370 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 10 client.16983 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 20 client.16983 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b2eab40 0x555a8b2ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 4 client.16983 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a8b1d4d70 con 0x555a8b2eab40 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b2eab40 0x555a8b2ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd8f7fe640 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b2eab40 0x555a8b2ecf60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c60599c81ccf33 server_cookie=9976b97c1d3ebc8 in_seq=0 out_seq=0 2024-09-19T09:41:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.340+0000 7fdd6effd640 10 client.16983 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd6effd640 1 -- 172.21.15.28:0/1940864875 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a8b1d4d70 con 0x555a8b2eab40 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd6effd640 10 client.16983 handle_command_reply: tid=0 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 client.16983 shutdown 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 msgr2=0x555a8b279ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdd84022880 tx=0x7fdd84009ac0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 shutdown_connections 2024-09-19T09:41:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555a8b2eab40 0x555a8b2ecf60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a8b26f370 0x555a8b279ac0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a8b26a2e0 0x555a8b278ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.753+0000 7fdd98caf740 1 --2- 172.21.15.28:0/1940864875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a8b2699a0 0x555a8b278570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd6effd640 0 client.16983 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 >> 172.21.15.28:0/1940864875 conn(0x555a8b265010 msgr2=0x555a8b2639a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 shutdown_connections 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd98caf740 1 -- 172.21.15.28:0/1940864875 wait complete. 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd98caf740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T09:41:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:41:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:41:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:41:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:41:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 msgr2=0x7fdd780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 0x7fdd780a5700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdd880bd260 tx=0x7fdd880882d0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 msgr2=0x7fdd9019f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdd8802f540 tx=0x7fdd88031750 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 shutdown_connections 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fdd780a3240 0x7fdd780a5700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd90101c30 0x7fdd9019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd901025f0 0x7fdd9019f180 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 --2- 172.21.15.28:0/3466108577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90100a40 0x7fdd9006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.754+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 >> 172.21.15.28:0/3466108577 conn(0x7fdd900fc1c0 msgr2=0x7fdd90104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.755+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 shutdown_connections 2024-09-19T09:41:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:23.755+0000 7fdd95f1a640 1 -- 172.21.15.28:0/3466108577 wait complete. 2024-09-19T09:41:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:23 smithi028 ceph-mon[30972]: pgmap v601: 97 pgs: 97 active+clean; 375 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-09-19T09:41:23.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:23 smithi078 ceph-mon[35619]: pgmap v601: 97 pgs: 97 active+clean; 375 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-09-19T09:41:23.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:23 smithi038 ceph-mon[35271]: pgmap v601: 97 pgs: 97 active+clean; 375 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-09-19T09:41:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:25 smithi028 ceph-mon[30972]: pgmap v602: 97 pgs: 97 active+clean; 362 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 525 op/s 2024-09-19T09:41:25.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:25 smithi078 ceph-mon[35619]: pgmap v602: 97 pgs: 97 active+clean; 362 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 525 op/s 2024-09-19T09:41:25.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:25 smithi038 ceph-mon[35271]: pgmap v602: 97 pgs: 97 active+clean; 362 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 525 op/s 2024-09-19T09:41:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:27 smithi028 ceph-mon[30972]: pgmap v603: 97 pgs: 97 active+clean; 346 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-19T09:41:27.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:27 smithi078 ceph-mon[35619]: pgmap v603: 97 pgs: 97 active+clean; 346 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-19T09:41:27.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:27 smithi038 ceph-mon[35271]: pgmap v603: 97 pgs: 97 active+clean; 346 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-09-19T09:41:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:29 smithi028 ceph-mon[30972]: pgmap v604: 97 pgs: 97 active+clean; 337 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2024-09-19T09:41:29.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:29 smithi078 ceph-mon[35619]: pgmap v604: 97 pgs: 97 active+clean; 337 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2024-09-19T09:41:29.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:29 smithi038 ceph-mon[35271]: pgmap v604: 97 pgs: 97 active+clean; 337 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2024-09-19T09:41:31.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:31 smithi028 ceph-mon[30972]: pgmap v605: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-09-19T09:41:31.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:31 smithi078 ceph-mon[35619]: pgmap v605: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-09-19T09:41:31.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:31 smithi038 ceph-mon[35271]: pgmap v605: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-09-19T09:41:33.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:33 smithi028 ceph-mon[30972]: pgmap v606: 97 pgs: 97 active+clean; 306 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 381 op/s 2024-09-19T09:41:33.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:33 smithi078 ceph-mon[35619]: pgmap v606: 97 pgs: 97 active+clean; 306 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 381 op/s 2024-09-19T09:41:33.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:33 smithi038 ceph-mon[35271]: pgmap v606: 97 pgs: 97 active+clean; 306 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 381 op/s 2024-09-19T09:41:35.727 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:35 smithi038 ceph-mon[35271]: pgmap v607: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-19T09:41:35.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:35 smithi028 ceph-mon[30972]: pgmap v607: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-19T09:41:35.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:35 smithi078 ceph-mon[35619]: pgmap v607: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-19T09:41:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 -- 172.21.15.28:0/1212104669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0104fb0 msgr2=0x7f7cc01073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/1212104669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0104fb0 0x7f7cc01073a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009900 tx=0x7f7cb002a410 comp rx=0 tx=0).stop 2024-09-19T09:41:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 -- 172.21.15.28:0/1212104669 shutdown_connections 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/1212104669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc01078e0 0x7f7cc0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/1212104669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0104fb0 0x7f7cc01073a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/1212104669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc0102680 0x7f7cc0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 -- 172.21.15.28:0/1212104669 >> 172.21.15.28:0/1212104669 conn(0x7f7cc00fc1c0 msgr2=0x7f7cc00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 -- 172.21.15.28:0/1212104669 shutdown_connections 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.847+0000 7f7cc73c4640 1 -- 172.21.15.28:0/1212104669 wait complete. 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 Processor -- start 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 -- start start 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc0104fb0 0x7f7cc0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc01078e0 0x7f7cc019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010d7c0 con 0x7f7cc0102680 2024-09-19T09:41:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010d640 con 0x7f7cc01078e0 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc73c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010d940 con 0x7f7cc0104fb0 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47112/0 (socket says 172.21.15.28:47112) 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 -- 172.21.15.28:0/3001743808 learned_addr learned my addr 172.21.15.28:0/3001743808 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc495f640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc0104fb0 0x7f7cc0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5961640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc01078e0 0x7f7cc019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 -- 172.21.15.28:0/3001743808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc0104fb0 msgr2=0x7f7cc0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc0104fb0 0x7f7cc0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 -- 172.21.15.28:0/3001743808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc01078e0 msgr2=0x7f7cc019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc01078e0 0x7f7cc019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.848+0000 7f7cc5160640 1 -- 172.21.15.28:0/3001743808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc019faf0 con 0x7f7cc0102680 2024-09-19T09:41:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00e9f0 tx=0x7f7cbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cb67fc640 1 -- 172.21.15.28:0/3001743808 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00cd80 con 0x7f7cc0102680 2024-09-19T09:41:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cb67fc640 1 -- 172.21.15.28:0/3001743808 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cbc010910 con 0x7f7cc0102680 2024-09-19T09:41:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cb67fc640 1 -- 172.21.15.28:0/3001743808 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc018990 con 0x7f7cc0102680 2024-09-19T09:41:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc019fde0 con 0x7f7cc0102680 2024-09-19T09:41:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.849+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01a01f0 con 0x7f7cc0102680 2024-09-19T09:41:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.851+0000 7f7cb67fc640 1 -- 172.21.15.28:0/3001743808 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7cbc010ac0 con 0x7f7cc0102680 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.851+0000 7f7cca23b740 1 Processor -- start 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.851+0000 7f7cca23b740 1 -- start start 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 0x555adbb3bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3c2e0 0x555adbb40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb43830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555adba9f2f0 con 0x555adbb3c2e0 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555adba47270 con 0x555adbb41370 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cca23b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555adba9b2b0 con 0x555adbb3b9a0 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc5961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 0x555adbb3bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3c2e0 0x555adbb40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb43830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc5961640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 0x555adbb3bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:40976/0 (socket says 172.21.15.28:40976) 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc5961640 1 -- 172.21.15.28:0/47591469 learned_addr learned my addr 172.21.15.28:0/47591469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cb67fc640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 0x7f7cac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc5961640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 0x7f7cac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 -- 172.21.15.28:0/47591469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 msgr2=0x555adbb3bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 0x555adbb3bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 -- 172.21.15.28:0/47591469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3c2e0 msgr2=0x555adbb40da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3c2e0 0x555adbb40da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cc495f640 1 -- 172.21.15.28:0/47591469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adba46340 con 0x555adbb41370 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.852+0000 7f7cb67fc640 1 -- 172.21.15.28:0/3001743808 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f7cbc0cfbb0 con 0x7f7cc0102680 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cc495f640 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb43830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009230 tx=0x7f7cb0005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7de873b65a015d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cc5961640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 0x7f7cac0a5780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009fd0 tx=0x7f7cb8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cab7fe640 1 -- 172.21.15.28:0/47591469 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0040070 con 0x555adbb41370 2024-09-19T09:41:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cab7fe640 1 -- 172.21.15.28:0/47591469 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb0004030 con 0x555adbb41370 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cab7fe640 1 -- 172.21.15.28:0/47591469 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0035040 con 0x555adbb41370 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cca23b740 1 -- 172.21.15.28:0/47591469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 msgr2=0x555adbb43830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.853+0000 7f7cca23b740 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb43830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009230 tx=0x7f7cb0005e20 comp rx=0 tx=0).stop 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- 172.21.15.28:0/47591469 shutdown_connections 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb43830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3c2e0 0x555adbb40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- 172.21.15.28:0/47591469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3b9a0 0x555adbb3bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- 172.21.15.28:0/47591469 >> 172.21.15.28:0/47591469 conn(0x555adbb37010 msgr2=0x555adbb37410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- 172.21.15.28:0/47591469 shutdown_connections 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- 172.21.15.28:0/47591469 wait complete. 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- start start 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 Processor -- start 2024-09-19T09:41:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb46040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555adbb442e0 con 0x555adbb3b9a0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555adba28290 con 0x555adbb41370 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555adbb44460 con 0x555adbb3c2e0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cca23b740 10 client.0 _ll_register_callbacks cb 0x555adba59140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.854+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47124/0 (socket says 172.21.15.28:47124) 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 -- 172.21.15.28:0/3102277963 learned_addr learned my addr 172.21.15.28:0/3102277963 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc495f640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb46040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5961640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 -- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 msgr2=0x555adbb46040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb46040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 -- 172.21.15.28:0/3102277963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 msgr2=0x555adbb4d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 -- 172.21.15.28:0/3102277963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adb7b2210 con 0x555adbb3b9a0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cc5160640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0cd680 tx=0x7f7cbc0d31f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd4a5a65c3f879c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc09bbb0 con 0x555adbb3b9a0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f7cbc0cdb40 con 0x555adbb3b9a0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb0004030 con 0x555adbb3b9a0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7cbc0984f0 con 0x555adbb3b9a0 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cca23b740 10 client.16995 fetch_fsmap learned FSMap version 46 2024-09-19T09:41:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555adb87e620 con 0x555adbb3b9a0 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.855+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb0004030 con 0x555adbb3b9a0 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 46) ==== 8390+0+0 (secure 0 0 0) 0x7f7cbc0a0160 con 0x555adbb3b9a0 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 10 client.16995 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 20 client.16995 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbb51830 0x555adbb53c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 4 client.16995 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555adbaaca20 con 0x555adbb51830 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cc495f640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbb51830 0x555adbb53c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7ca8ff9640 10 client.16995 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.856+0000 7f7cc495f640 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbb51830 0x555adbb53c20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6756f0975ee0a4 server_cookie=e8ba7f92b53498c in_seq=0 out_seq=0 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.857+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/3102277963 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555adbaaca20 con 0x555adbb51830 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.857+0000 7f7ca8ff9640 10 client.16995 handle_command_reply: tid=0 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.857+0000 7f7cca23b740 1 client.16995 shutdown 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 msgr2=0x555adbb45b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0cd680 tx=0x7f7cbc0d31f0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 shutdown_connections 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbb51830 0x555adbb53c20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 --2- 172.21.15.28:0/3102277963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45b00 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 >> 172.21.15.28:0/3102277963 conn(0x555adbb37010 msgr2=0x555adbb38820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7ca8ff9640 0 client.16995 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 shutdown_connections 2024-09-19T09:41:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 1 -- 172.21.15.28:0/3102277963 wait complete. 2024-09-19T09:41:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.858+0000 7f7cca23b740 20 client.16995 trim_cache size 0 max 16384 2024-09-19T09:41:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 Processor -- start 2024-09-19T09:41:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 -- start start 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb37ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01a8f50 con 0x555adbb3b9a0 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01aabb0 con 0x555adbb41370 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cca23b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01a96b0 con 0x555adbb3c2e0 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.872+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47140/0 (socket says 172.21.15.28:47140) 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 -- 172.21.15.28:0/650238854 learned_addr learned my addr 172.21.15.28:0/650238854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc495f640 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5961640 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 -- 172.21.15.28:0/650238854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 msgr2=0x555adbb4d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 -- 172.21.15.28:0/650238854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 msgr2=0x555adbb37ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb37ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 -- 172.21.15.28:0/650238854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adbb43c40 con 0x555adbb3b9a0 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7cc5160640 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00cfd0 tx=0x7f7cbc0cfe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75cc2ea5890ce80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7caaffd640 1 -- 172.21.15.28:0/650238854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0a0550 con 0x555adbb3b9a0 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7caaffd640 1 -- 172.21.15.28:0/650238854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cbc0a4e10 con 0x555adbb3b9a0 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.873+0000 7f7caaffd640 1 -- 172.21.15.28:0/650238854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc097fa0 con 0x555adbb3b9a0 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- 172.21.15.28:0/650238854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 msgr2=0x555adbb45fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00cfd0 tx=0x7f7cbc0cfe90 comp rx=0 tx=0).stop 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- 172.21.15.28:0/650238854 shutdown_connections 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb37ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- 172.21.15.28:0/650238854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb45fc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- 172.21.15.28:0/650238854 >> 172.21.15.28:0/650238854 conn(0x555adbb37010 msgr2=0x555adbb458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- 172.21.15.28:0/650238854 shutdown_connections 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- 172.21.15.28:0/650238854 wait complete. 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- start start 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 Processor -- start 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01aa4e0 con 0x555adbb3b9a0 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01a9de0 con 0x555adbb41370 2024-09-19T09:41:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01a6f30 con 0x555adbb3c2e0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cca23b740 10 client.0 _ll_register_callbacks cb 0x7f7cbc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.874+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47152/0 (socket says 172.21.15.28:47152) 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 -- 172.21.15.28:0/529928609 learned_addr learned my addr 172.21.15.28:0/529928609 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc495f640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5961640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 -- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 msgr2=0x555adbb4aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 -- 172.21.15.28:0/529928609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 msgr2=0x555adbb4ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 -- 172.21.15.28:0/529928609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adbb4c090 con 0x555adbb3b9a0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cc5160640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc023040 tx=0x7f7cbc09c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5567921f2fb7b9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0bc0b0 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f7cbc0a0170 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cbc0a4e10 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7cbc0a0550 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cca23b740 10 client.17001 fetch_fsmap learned FSMap version 46 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555adbbbc500 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.875+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cbc0a4e10 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 46) ==== 8390+0+0 (secure 0 0 0) 0x7f7cbc0a16e0 con 0x555adbb3b9a0 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 10 client.17001 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 20 client.17001 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbbbcb00 0x555adbbbef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 4 client.17001 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:41:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555adbaa6d70 con 0x555adbbbcb00 2024-09-19T09:41:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cc495f640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbbbcb00 0x555adbbbef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7cc495f640 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbbbcb00 0x555adbbbef20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b1a14520dfa64b server_cookie=979521c2a164b0c0 in_seq=0 out_seq=0 2024-09-19T09:41:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:36.876+0000 7f7ca8ff9640 10 client.17001 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.120+0000 7f7ca8ff9640 1 -- 172.21.15.28:0/529928609 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555adbaa6d70 con 0x555adbbbcb00 2024-09-19T09:41:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7ca8ff9640 10 client.17001 handle_command_reply: tid=0 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 client.17001 shutdown 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 msgr2=0x555adbb4a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc023040 tx=0x7f7cbc09c850 comp rx=0 tx=0).stop 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 shutdown_connections 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555adbb41370 0x555adbb4ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555adbbbcb00 0x555adbbbef20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555adbb3c2e0 0x555adbb4aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7ca8ff9640 0 client.17001 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 --2- 172.21.15.28:0/529928609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555adbb3b9a0 0x555adbb4a5a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 >> 172.21.15.28:0/529928609 conn(0x555adbb37010 msgr2=0x555adbb359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 shutdown_connections 2024-09-19T09:41:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 1 -- 172.21.15.28:0/529928609 wait complete. 2024-09-19T09:41:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.121+0000 7f7cca23b740 20 client.17001 trim_cache size 0 max 16384 2024-09-19T09:41:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:41:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:41:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:41:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 msgr2=0x7f7cac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 0x7f7cac0a5780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009fd0 tx=0x7f7cb8004ab0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 msgr2=0x7f7cc0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00e9f0 tx=0x7f7cbc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 shutdown_connections 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f7cac0a32c0 0x7f7cac0a5780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc01078e0 0x7f7cc019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cc0104fb0 0x7f7cc0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 --2- 172.21.15.28:0/3001743808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cc0102680 0x7f7cc0197ed0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 >> 172.21.15.28:0/3001743808 conn(0x7f7cc00fc1c0 msgr2=0x7f7cc0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 shutdown_connections 2024-09-19T09:41:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:37.122+0000 7f7cc73c4640 1 -- 172.21.15.28:0/3001743808 wait complete. 2024-09-19T09:41:37.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:37 smithi028 ceph-mon[30972]: pgmap v608: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 412 op/s 2024-09-19T09:41:37.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:37 smithi078 ceph-mon[35619]: pgmap v608: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 412 op/s 2024-09-19T09:41:37.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:37 smithi038 ceph-mon[35271]: pgmap v608: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 412 op/s 2024-09-19T09:41:39.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:39 smithi028 ceph-mon[30972]: pgmap v609: 97 pgs: 97 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2024-09-19T09:41:39.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:39 smithi078 ceph-mon[35619]: pgmap v609: 97 pgs: 97 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2024-09-19T09:41:39.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:39 smithi038 ceph-mon[35271]: pgmap v609: 97 pgs: 97 active+clean; 275 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2024-09-19T09:41:41.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:41 smithi028 ceph-mon[30972]: pgmap v610: 97 pgs: 97 active+clean; 247 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-09-19T09:41:41.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:41 smithi078 ceph-mon[35619]: pgmap v610: 97 pgs: 97 active+clean; 247 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-09-19T09:41:41.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:41 smithi038 ceph-mon[35271]: pgmap v610: 97 pgs: 97 active+clean; 247 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-09-19T09:41:43.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:43 smithi028 ceph-mon[30972]: pgmap v611: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-09-19T09:41:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:43 smithi078 ceph-mon[35619]: pgmap v611: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-09-19T09:41:43.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:43 smithi038 ceph-mon[35271]: pgmap v611: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-09-19T09:41:45.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:45 smithi028 ceph-mon[30972]: pgmap v612: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 493 op/s 2024-09-19T09:41:45.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:45 smithi078 ceph-mon[35619]: pgmap v612: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 493 op/s 2024-09-19T09:41:45.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:45 smithi038 ceph-mon[35271]: pgmap v612: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 493 op/s 2024-09-19T09:41:47.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:47 smithi028 ceph-mon[30972]: pgmap v613: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 451 op/s 2024-09-19T09:41:47.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:47 smithi078 ceph-mon[35619]: pgmap v613: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 451 op/s 2024-09-19T09:41:47.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:47 smithi038 ceph-mon[35271]: pgmap v613: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 451 op/s 2024-09-19T09:41:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:48 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:48.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:48 smithi028 ceph-mon[30972]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-09-19T09:41:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:48 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:48.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:48 smithi078 ceph-mon[35619]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-09-19T09:41:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:48 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:41:48.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:48 smithi038 ceph-mon[35271]: pgmap v614: 97 pgs: 97 active+clean; 207 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-09-19T09:41:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.210+0000 7f3d1a65e640 1 -- 172.21.15.28:0/770792013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 msgr2=0x7f3d14109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.210+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/770792013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d14109f80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ca0 tx=0x7f3d0c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- 172.21.15.28:0/770792013 shutdown_connections 2024-09-19T09:41:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/770792013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 0x7f3d141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/770792013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d14109f80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/770792013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- 172.21.15.28:0/770792013 >> 172.21.15.28:0/770792013 conn(0x7f3d140fd2e0 msgr2=0x7f3d140ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- 172.21.15.28:0/770792013 shutdown_connections 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- 172.21.15.28:0/770792013 wait complete. 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 Processor -- start 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- start start 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 0x7f3d14198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d1419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1410d7c0 con 0x7f3d141036f0 2024-09-19T09:41:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1410d640 con 0x7f3d14102d30 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.211+0000 7f3d1a65e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1410d940 con 0x7f3d14101b40 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d18bfb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d1419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:53522/0 (socket says 172.21.15.28:53522) 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 -- 172.21.15.28:0/3556241861 learned_addr learned my addr 172.21.15.28:0/3556241861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 0x7f3d14198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 -- 172.21.15.28:0/3556241861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 msgr2=0x7f3d14198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 0x7f3d14198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 -- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 msgr2=0x7f3d1419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d1419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 -- 172.21.15.28:0/3556241861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1419fa00 con 0x7f3d14101b40 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d1419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:41:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d13fff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9c0 tx=0x7f3d0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d11ffb640 1 -- 172.21.15.28:0/3556241861 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400cd80 con 0x7f3d14101b40 2024-09-19T09:41:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1419fcf0 con 0x7f3d14101b40 2024-09-19T09:41:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d11ffb640 1 -- 172.21.15.28:0/3556241861 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d040035f0 con 0x7f3d14101b40 2024-09-19T09:41:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d11ffb640 1 -- 172.21.15.28:0/3556241861 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04010690 con 0x7f3d14101b40 2024-09-19T09:41:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.212+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d141a0190 con 0x7f3d14101b40 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d11ffb640 1 -- 172.21.15.28:0/3556241861 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3d040037a0 con 0x7f3d14101b40 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 Processor -- start 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 -- start start 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1af7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1afa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555cc1a562f0 con 0x555cc1af29a0 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555cc19fe270 con 0x555cc1af32e0 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555cc1a522b0 con 0x555cc1af8370 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45610/0 (socket says 172.21.15.28:45610) 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 -- 172.21.15.28:0/477682729 learned_addr learned my addr 172.21.15.28:0/477682729 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1afa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d13fff640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1af7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 -- 172.21.15.28:0/477682729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 msgr2=0x555cc1afa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1afa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 -- 172.21.15.28:0/477682729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 msgr2=0x555cc1af7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1af7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d18bfb640 1 -- 172.21.15.28:0/477682729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc19fd340 con 0x555cc1af29a0 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.215+0000 7f3d13fff640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1af7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d11ffb640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 0x7f3d000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ca0 tx=0x7f3d0c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41f72468893d16b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d13fff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 0x7f3d000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d0a7fc640 1 -- 172.21.15.28:0/477682729 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c049070 con 0x555cc1af29a0 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d11ffb640 1 -- 172.21.15.28:0/3556241861 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f3d040cfc30 con 0x7f3d14101b40 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d0a7fc640 1 -- 172.21.15.28:0/477682729 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d0c036040 con 0x555cc1af29a0 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.216+0000 7f3d0a7fc640 1 -- 172.21.15.28:0/477682729 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c030820 con 0x555cc1af29a0 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/477682729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 msgr2=0x555cc1af2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d13fff640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 0x7f3d000a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d040cdad0 tx=0x7f3d04003f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ca0 tx=0x7f3d0c004a80 comp rx=0 tx=0).stop 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/477682729 shutdown_connections 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1af7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1afa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/477682729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1af2da0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/477682729 >> 172.21.15.28:0/477682729 conn(0x555cc1aee010 msgr2=0x555cc1aee410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/477682729 shutdown_connections 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/477682729 wait complete. 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 -- start start 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 Processor -- start 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1afc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.217+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1b03f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555cc1afb160 con 0x555cc1af32e0 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555cc19df290 con 0x555cc1af8370 2024-09-19T09:41:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555cc1afb2e0 con 0x555cc1af29a0 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 10 client.0 _ll_register_callbacks cb 0x555cc1a10140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45624/0 (socket says 172.21.15.28:45624) 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 -- 172.21.15.28:0/3907165712 learned_addr learned my addr 172.21.15.28:0/3907165712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1afc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1b03f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 -- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 msgr2=0x555cc1afc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1afc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 -- 172.21.15.28:0/3907165712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 msgr2=0x555cc1b03f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1b03f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 -- 172.21.15.28:0/3907165712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc1769210 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1b03f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00bad0 tx=0x7f3cfc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baae2c3216f96ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011440 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f3cfc0115e0 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c036040 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc011820 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 10 client.17013 fetch_fsmap learned FSMap version 47 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.218+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555cc1835620 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d0c036040 con 0x555cc1af32e0 2024-09-19T09:41:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f3cfc026b40 con 0x555cc1af32e0 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 10 client.17013 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 20 client.17013 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b08730 0x555cc1b0ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 4 client.17013 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc1a63a20 con 0x555cc1b08730 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b08730 0x555cc1b0ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b08730 0x555cc1b0ab40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779366307de81e6b server_cookie=c72e060ca06981e9 in_seq=0 out_seq=0 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.219+0000 7f3cf37fe640 10 client.17013 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3cf37fe640 1 -- 172.21.15.28:0/3907165712 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cc1a63a20 con 0x555cc1b08730 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3cf37fe640 10 client.17013 handle_command_reply: tid=0 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 client.17013 shutdown 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 msgr2=0x555cc1afcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00bad0 tx=0x7f3cfc00bf90 comp rx=0 tx=0).stop 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 shutdown_connections 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1b03f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b08730 0x555cc1b0ab40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1afcf30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/3907165712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1afc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3cf37fe640 0 client.17013 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.220+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 >> 172.21.15.28:0/3907165712 conn(0x555cc1aee010 msgr2=0x555cc1aef820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.221+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 shutdown_connections 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.221+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/3907165712 wait complete. 2024-09-19T09:41:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.221+0000 7f3d1d3fc740 20 client.17013 trim_cache size 0 max 16384 2024-09-19T09:41:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 Processor -- start 2024-09-19T09:41:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 -- start start 2024-09-19T09:41:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1af2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1aef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141a8e70 con 0x555cc1af32e0 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141aaaa0 con 0x555cc1af8370 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141a95d0 con 0x555cc1af29a0 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45638/0 (socket says 172.21.15.28:45638) 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d0bfff640 1 -- 172.21.15.28:0/753909717 learned_addr learned my addr 172.21.15.28:0/753909717 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.236+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1af2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1aef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 -- 172.21.15.28:0/753909717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 msgr2=0x555cc1af2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1af2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 -- 172.21.15.28:0/753909717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 msgr2=0x555cc1aef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1aef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 -- 172.21.15.28:0/753909717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc1af37c0 con 0x555cc1af32e0 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc003a40 tx=0x7f3cfc0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e00c79d1286a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d09ffb640 1 -- 172.21.15.28:0/753909717 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc01b560 con 0x555cc1af32e0 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d09ffb640 1 -- 172.21.15.28:0/753909717 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc01b700 con 0x555cc1af32e0 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.237+0000 7f3d09ffb640 1 -- 172.21.15.28:0/753909717 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc005040 con 0x555cc1af32e0 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/753909717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 msgr2=0x555cc1aed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc003a40 tx=0x7f3cfc0044a0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/753909717 shutdown_connections 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af8370 0x555cc1aef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af32e0 0x555cc1aed0b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/753909717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af29a0 0x555cc1af2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/753909717 >> 172.21.15.28:0/753909717 conn(0x555cc1aee010 msgr2=0x555cc1aec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/753909717 shutdown_connections 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/753909717 wait complete. 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- start start 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 Processor -- start 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1b6f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1b76150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141aa3d0 con 0x555cc1af29a0 2024-09-19T09:41:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141a9d00 con 0x555cc1af32e0 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.238+0000 7f3d1d3fc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141a6e50 con 0x555cc1af8370 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d1d3fc740 10 client.0 _ll_register_callbacks cb 0x7f3cfc004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d18bfb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d18bfb640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45646/0 (socket says 172.21.15.28:45646) 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d18bfb640 1 -- 172.21.15.28:0/4003223048 learned_addr learned my addr 172.21.15.28:0/4003223048 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1b6f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1b76150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 -- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 msgr2=0x555cc1b76150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1b76150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 -- 172.21.15.28:0/4003223048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 msgr2=0x555cc1b6ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 -- 172.21.15.28:0/4003223048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc1b76760 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d0bfff640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1b6f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc01b490 tx=0x7f3cfc011670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=268e739c8a572b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc03c060 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc01b700 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.239+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f3cfc011bd0 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc011dd0 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 10 client.26540 fetch_fsmap learned FSMap version 47 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555cc1b77a70 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc01b700 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f3cfc004640 con 0x555cc1af32e0 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 10 client.26540 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T09:41:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 20 client.26540 populate_metadata read hostname 'smithi028' 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b783e0 0x555cc1b7a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 4 client.26540 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cc1a5dd70 con 0x555cc1b783e0 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.240+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b783e0 0x555cc1b7a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.241+0000 7f3cf37fe640 10 client.26540 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.241+0000 7f3d18bfb640 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b783e0 0x555cc1b7a800 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963426fa9b07d986 server_cookie=c52ce7c2f997221c in_seq=0 out_seq=0 2024-09-19T09:41:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.739+0000 7f3cf37fe640 1 -- 172.21.15.28:0/4003223048 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cc1a5dd70 con 0x555cc1b783e0 2024-09-19T09:41:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.739+0000 7f3cf37fe640 10 client.26540 handle_command_reply: tid=0 2024-09-19T09:41:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.739+0000 7f3d1d3fc740 1 client.26540 shutdown 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 msgr2=0x555cc1b6f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1b6f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc01b490 tx=0x7f3cfc011670 comp rx=0 tx=0).stop 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 shutdown_connections 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555cc1af32e0 0x555cc1b6f110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x555cc1b783e0 0x555cc1b7a800 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555cc1af8370 0x555cc1b76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3cf37fe640 0 client.26540 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 --2- 172.21.15.28:0/4003223048 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555cc1af29a0 0x555cc1b6ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 >> 172.21.15.28:0/4003223048 conn(0x555cc1aee010 msgr2=0x555cc1aed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 shutdown_connections 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 1 -- 172.21.15.28:0/4003223048 wait complete. 2024-09-19T09:41:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.740+0000 7f3d1d3fc740 20 client.26540 trim_cache size 0 max 16384 2024-09-19T09:41:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:41:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:41:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:41:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 msgr2=0x7f3d000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 0x7f3d000a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d040cdad0 tx=0x7f3d04003f10 comp rx=0 tx=0).stop 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 msgr2=0x7f3d14197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9c0 tx=0x7f3d0400ee80 comp rx=0 tx=0).stop 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 shutdown_connections 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3d000a3240 0x7f3d000a5700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d14102d30 0x7f3d14198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d141036f0 0x7f3d1419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 --2- 172.21.15.28:0/3556241861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d14101b40 0x7f3d14197de0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:41:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 >> 172.21.15.28:0/3556241861 conn(0x7f3d140fd2e0 msgr2=0x7f3d14106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:41:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 shutdown_connections 2024-09-19T09:41:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:41:50.741+0000 7f3d1a65e640 1 -- 172.21.15.28:0/3556241861 wait complete. 2024-09-19T09:41:50.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:50 smithi038 ceph-mon[35271]: pgmap v615: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T09:41:51.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:50 smithi028 ceph-mon[30972]: pgmap v615: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T09:41:51.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:50 smithi078 ceph-mon[35619]: pgmap v615: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-09-19T09:41:52.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:52 smithi038 ceph-mon[35271]: pgmap v616: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-19T09:41:53.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:52 smithi028 ceph-mon[30972]: pgmap v616: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-19T09:41:53.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:52 smithi078 ceph-mon[35619]: pgmap v616: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: *** Caught signal (Segmentation fault) ** 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: in thread 7f5855ed6640 thread_name:md_log_replay 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f58632ad6f0] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2: (Locker::mark_updated_scatterlock(ScatterLock*)+0x2e) [0x5609b5a3632e] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 3: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x1e6b) [0x5609b58ad6ab] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 4: (EUpdate::replay(MDSRank*)+0x5d) [0x5609b58b92ad] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 5: (MDLog::_replay_thread()+0x759) [0x5609b5b2e709] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 6: /usr/bin/ceph-mds(+0x44bb11) [0x5609b5b1ab11] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 7: /lib64/libc.so.6(+0x89c52) [0x7f58632f8c52] 2024-09-19T09:41:53.515 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 8: /lib64/libc.so.6(+0x10ec80) [0x7f586337dc80] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.240+0000 7f5855ed6640 -1 *** Caught signal (Segmentation fault) ** 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: in thread 7f5855ed6640 thread_name:md_log_replay 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f58632ad6f0] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2: (Locker::mark_updated_scatterlock(ScatterLock*)+0x2e) [0x5609b5a3632e] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 3: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x1e6b) [0x5609b58ad6ab] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 4: (EUpdate::replay(MDSRank*)+0x5d) [0x5609b58b92ad] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 5: (MDLog::_replay_thread()+0x759) [0x5609b5b2e709] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 6: /usr/bin/ceph-mds(+0x44bb11) [0x5609b5b1ab11] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 7: /lib64/libc.so.6(+0x89c52) [0x7f58632f8c52] 2024-09-19T09:41:53.516 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 8: /lib64/libc.so.6(+0x10ec80) [0x7f586337dc80] 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 0> 2024-09-19T09:41:53.240+0000 7f5855ed6640 -1 *** Caught signal (Segmentation fault) ** 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: in thread 7f5855ed6640 thread_name:md_log_replay 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f58632ad6f0] 2024-09-19T09:41:53.517 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2: (Locker::mark_updated_scatterlock(ScatterLock*)+0x2e) [0x5609b5a3632e] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 3: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x1e6b) [0x5609b58ad6ab] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 4: (EUpdate::replay(MDSRank*)+0x5d) [0x5609b58b92ad] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 5: (MDLog::_replay_thread()+0x759) [0x5609b5b2e709] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 6: /usr/bin/ceph-mds(+0x44bb11) [0x5609b5b1ab11] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 7: /lib64/libc.so.6(+0x89c52) [0x7f58632f8c52] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 8: /lib64/libc.so.6(+0x10ec80) [0x7f586337dc80] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 0> 2024-09-19T09:41:53.240+0000 7f5855ed6640 -1 *** Caught signal (Segmentation fault) ** 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: in thread 7f5855ed6640 thread_name:md_log_replay 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f58632ad6f0] 2024-09-19T09:41:53.518 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2: (Locker::mark_updated_scatterlock(ScatterLock*)+0x2e) [0x5609b5a3632e] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 3: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x1e6b) [0x5609b58ad6ab] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 4: (EUpdate::replay(MDSRank*)+0x5d) [0x5609b58b92ad] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 5: (MDLog::_replay_thread()+0x759) [0x5609b5b2e709] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 6: /usr/bin/ceph-mds(+0x44bb11) [0x5609b5b1ab11] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 7: /lib64/libc.so.6(+0x89c52) [0x7f58632f8c52] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 8: /lib64/libc.so.6(+0x10ec80) [0x7f586337dc80] 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[96939]: 2024-09-19T09:41:53.519 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:53 smithi028 podman[100436]: 2024-09-19 09:41:53.297863894 +0000 UTC m=+0.018826411 container died 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:41:54.514 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:54 smithi028 podman[100436]: 2024-09-19 09:41:54.261801084 +0000 UTC m=+0.982763588 container remove 7ae61e3bf86b67571e965c44c151dff5b48aeaff24dd90bef591fcb31c473e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:41:54.514 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:54 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-09-19T09:41:54.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:54 smithi038 ceph-mon[35271]: pgmap v617: 97 pgs: 97 active+clean; 153 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 482 op/s 2024-09-19T09:41:55.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:54 smithi028 ceph-mon[30972]: pgmap v617: 97 pgs: 97 active+clean; 153 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 482 op/s 2024-09-19T09:41:55.014 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:54 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-09-19T09:41:55.015 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:41:54 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Consumed 8.595s CPU time. 2024-09-19T09:41:55.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:54 smithi078 ceph-mon[35619]: pgmap v617: 97 pgs: 97 active+clean; 153 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 482 op/s 2024-09-19T09:41:56.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:56 smithi038 ceph-mon[35271]: pgmap v618: 97 pgs: 97 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:41:57.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:56 smithi028 ceph-mon[30972]: pgmap v618: 97 pgs: 97 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:41:57.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:56 smithi078 ceph-mon[35619]: pgmap v618: 97 pgs: 97 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:41:58.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:41:58 smithi038 ceph-mon[35271]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-09-19T09:41:59.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:41:58 smithi028 ceph-mon[30972]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-09-19T09:41:59.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:41:58 smithi078 ceph-mon[35619]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-09-19T09:42:00.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:00 smithi038 ceph-mon[35271]: pgmap v620: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 524 op/s 2024-09-19T09:42:01.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:00 smithi028 ceph-mon[30972]: pgmap v620: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 524 op/s 2024-09-19T09:42:01.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:00 smithi078 ceph-mon[35619]: pgmap v620: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 524 op/s 2024-09-19T09:42:02.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:02 smithi038 ceph-mon[35271]: pgmap v621: 97 pgs: 97 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-09-19T09:42:03.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:02 smithi028 ceph-mon[30972]: pgmap v621: 97 pgs: 97 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-09-19T09:42:03.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:02 smithi078 ceph-mon[35619]: pgmap v621: 97 pgs: 97 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-09-19T09:42:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 -- 172.21.15.28:0/1079498668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0101c60 msgr2=0x7f05d01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/1079498668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0101c60 0x7f05d01020e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c002a410 comp rx=0 tx=0).stop 2024-09-19T09:42:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 -- 172.21.15.28:0/1079498668 shutdown_connections 2024-09-19T09:42:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/1079498668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0101c60 0x7f05d01020e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/1079498668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d0108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/1079498668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0100a70 0x7f05d0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.836+0000 7f05d7ed7640 1 -- 172.21.15.28:0/1079498668 >> 172.21.15.28:0/1079498668 conn(0x7f05d00fc1f0 msgr2=0x7f05d00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.837+0000 7f05d7ed7640 1 -- 172.21.15.28:0/1079498668 shutdown_connections 2024-09-19T09:42:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.837+0000 7f05d7ed7640 1 -- 172.21.15.28:0/1079498668 wait complete. 2024-09-19T09:42:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 Processor -- start 2024-09-19T09:42:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 -- start start 2024-09-19T09:42:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0100a70 0x7f05d0195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0101c60 0x7f05d0196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.838+0000 7f05d7ed7640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010b670 con 0x7f05d0101c60 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d7ed7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010b4f0 con 0x7f05d0100a70 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d7ed7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010b7f0 con 0x7f05d0102620 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:44364/0 (socket says 172.21.15.28:44364) 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 -- 172.21.15.28:0/713047380 learned_addr learned my addr 172.21.15.28:0/713047380 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d5c73640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0100a70 0x7f05d0195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d5472640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0101c60 0x7f05d0196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 -- 172.21.15.28:0/713047380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0100a70 msgr2=0x7f05d0195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0100a70 0x7f05d0195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 -- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0101c60 msgr2=0x7f05d0196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0101c60 0x7f05d0196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 -- 172.21.15.28:0/713047380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d006bd40 con 0x7f05d0102620 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.839+0000 7f05d6474640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05c8003fd0 tx=0x7f05c800fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.841+0000 7f05c6ffd640 1 -- 172.21.15.28:0/713047380 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004830 con 0x7f05d0102620 2024-09-19T09:42:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.841+0000 7f05c6ffd640 1 -- 172.21.15.28:0/713047380 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c8005020 con 0x7f05d0102620 2024-09-19T09:42:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.841+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d006c030 con 0x7f05d0102620 2024-09-19T09:42:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.841+0000 7f05c6ffd640 1 -- 172.21.15.28:0/713047380 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c803a670 con 0x7f05d0102620 2024-09-19T09:42:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.841+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d006c580 con 0x7f05d0102620 2024-09-19T09:42:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.845+0000 7f05dac70740 1 Processor -- start 2024-09-19T09:42:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.845+0000 7f05c6ffd640 1 -- 172.21.15.28:0/713047380 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f05c8041020 con 0x7f05d0102620 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 -- start start 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c39da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c3c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ef99b982f0 con 0x55ef99c3a370 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef99b40270 con 0x55ef99c352e0 2024-09-19T09:42:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05dac70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef99b942b0 con 0x55ef99c349a0 2024-09-19T09:42:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05d5c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c39da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05d5472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05d5472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:44372/0 (socket says 172.21.15.28:44372) 2024-09-19T09:42:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05d5472640 1 -- 172.21.15.28:0/1515525316 learned_addr learned my addr 172.21.15.28:0/1515525316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.846+0000 7f05d5472640 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c3c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05c6ffd640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 0x7f05bc0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 -- 172.21.15.28:0/1515525316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 msgr2=0x55ef99c39da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c39da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 -- 172.21.15.28:0/1515525316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 msgr2=0x55ef99c3c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c3c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 -- 172.21.15.28:0/1515525316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef99b3f340 con 0x55ef99c349a0 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05c6ffd640 1 -- 172.21.15.28:0/713047380 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f05c80f5570 con 0x7f05d0102620 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5c73640 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d6474640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 0x7f05bc0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d5472640 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009230 tx=0x7f05c0031440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3792e612bc32f6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05bbfff640 1 -- 172.21.15.28:0/1515525316 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0035040 con 0x55ef99c349a0 2024-09-19T09:42:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05bbfff640 1 -- 172.21.15.28:0/1515525316 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c0031d90 con 0x55ef99c349a0 2024-09-19T09:42:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05bbfff640 1 -- 172.21.15.28:0/1515525316 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0003d80 con 0x55ef99c349a0 2024-09-19T09:42:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.847+0000 7f05d6474640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 0x7f05bc0a5850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f05c8003bf0 tx=0x7f05c8003ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 -- 172.21.15.28:0/1515525316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 msgr2=0x55ef99c34da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009230 tx=0x7f05c0031440 comp rx=0 tx=0).stop 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 -- 172.21.15.28:0/1515525316 shutdown_connections 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c3c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 --2- 172.21.15.28:0/1515525316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c34da0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.848+0000 7f05dac70740 1 -- 172.21.15.28:0/1515525316 >> 172.21.15.28:0/1515525316 conn(0x55ef99c30010 msgr2=0x55ef99c30410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.849+0000 7f05dac70740 1 -- 172.21.15.28:0/1515525316 shutdown_connections 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.849+0000 7f05dac70740 1 -- 172.21.15.28:0/1515525316 wait complete. 2024-09-19T09:42:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.849+0000 7f05dac70740 1 -- start start 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 Processor -- start 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ef99c3d2e0 con 0x55ef99c3a370 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef99b21290 con 0x55ef99c352e0 2024-09-19T09:42:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef99c3d460 con 0x55ef99c349a0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05dac70740 10 client.0 _ll_register_callbacks cb 0x55ef99b52140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05d5c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05d5c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:49796/0 (socket says 172.21.15.28:49796) 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05d5c73640 1 -- 172.21.15.28:0/2008550535 learned_addr learned my addr 172.21.15.28:0/2008550535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05d5472640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.850+0000 7f05d5472640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 -- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 msgr2=0x55ef99c3ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c3ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 -- 172.21.15.28:0/2008550535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 msgr2=0x55ef99c46170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c46170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 -- 172.21.15.28:0/2008550535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef998ab210 con 0x55ef99c352e0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05d5c73640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009620 tx=0x7f05cc009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac85b27c4934278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.851+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc01d030 con 0x55ef99c352e0 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.852+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mon.2 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f05cc004240 con 0x55ef99c352e0 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.852+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c0031d90 con 0x55ef99c352e0 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.852+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f05cc004440 con 0x55ef99c352e0 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.852+0000 7f05dac70740 10 client.26552 fetch_fsmap learned FSMap version 47 2024-09-19T09:42:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.852+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ef99977620 con 0x55ef99c352e0 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mon.2 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c0031d90 con 0x55ef99c352e0 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mon.2 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f05cc00c070 con 0x55ef99c352e0 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 10 client.26552 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 20 client.26552 populate_metadata read hostname 'smithi028' 2024-09-19T09:42:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99c4ada0 0x55ef99c4d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 4 client.26552 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.853+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef99ba5a20 con 0x55ef99c4ada0 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.854+0000 7f05d5472640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99c4ada0 0x55ef99c4d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.854+0000 7f05b97fa640 10 client.26552 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.854+0000 7f05d5472640 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99c4ada0 0x55ef99c4d1e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dc876c950f69ce server_cookie=d3bf76ceaf276ce1 in_seq=0 out_seq=0 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.855+0000 7f05b97fa640 1 -- 172.21.15.28:0/2008550535 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef99ba5a20 con 0x55ef99c4ada0 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.855+0000 7f05b97fa640 10 client.26552 handle_command_reply: tid=0 2024-09-19T09:42:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.855+0000 7f05dac70740 1 client.26552 shutdown 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 msgr2=0x55ef99c3f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009620 tx=0x7f05cc009ae0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 shutdown_connections 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c3f130 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99c4ada0 0x55ef99c4d1e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c3a370 0x55ef99c46170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 --2- 172.21.15.28:0/2008550535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c349a0 0x55ef99c3ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 >> 172.21.15.28:0/2008550535 conn(0x55ef99c30010 msgr2=0x55ef99c31820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05b97fa640 0 client.26552 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 shutdown_connections 2024-09-19T09:42:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 1 -- 172.21.15.28:0/2008550535 wait complete. 2024-09-19T09:42:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.856+0000 7f05dac70740 20 client.26552 trim_cache size 0 max 16384 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 Processor -- start 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 -- start start 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99c30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05d019e5a0 con 0x55ef99c349a0 2024-09-19T09:42:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01a01d0 con 0x55ef99c352e0 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05dac70740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05d019ed00 con 0x55ef99c3a370 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05d5472640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05d5472640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37840/0 (socket says 172.21.15.28:37840) 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05d5472640 1 -- 172.21.15.28:0/1786755407 learned_addr learned my addr 172.21.15.28:0/1786755407 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.880+0000 7f05d5c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 -- 172.21.15.28:0/1786755407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 msgr2=0x55ef99c30930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99c30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 -- 172.21.15.28:0/1786755407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 msgr2=0x55ef99c46130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c46130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 -- 172.21.15.28:0/1786755407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef99c2fe30 con 0x55ef99c349a0 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05d5472640 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005ab0 tx=0x7f05c0004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed97f9c772dc319 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05bb7fe640 1 -- 172.21.15.28:0/1786755407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0035040 con 0x55ef99c349a0 2024-09-19T09:42:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05bb7fe640 1 -- 172.21.15.28:0/1786755407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c00316b0 con 0x55ef99c349a0 2024-09-19T09:42:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.881+0000 7f05bb7fe640 1 -- 172.21.15.28:0/1786755407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c00037f0 con 0x55ef99c349a0 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- 172.21.15.28:0/1786755407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 msgr2=0x55ef99c303f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005ab0 tx=0x7f05c0004210 comp rx=0 tx=0).stop 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- 172.21.15.28:0/1786755407 shutdown_connections 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c46130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99c30930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 --2- 172.21.15.28:0/1786755407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c303f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- 172.21.15.28:0/1786755407 >> 172.21.15.28:0/1786755407 conn(0x55ef99c30010 msgr2=0x55ef99c36520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- 172.21.15.28:0/1786755407 shutdown_connections 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- 172.21.15.28:0/1786755407 wait complete. 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 -- start start 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.882+0000 7f05dac70740 1 Processor -- start 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99cb3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05d5472640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05d5472640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37852/0 (socket says 172.21.15.28:37852) 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05d5c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05d5472640 1 -- 172.21.15.28:0/931792095 learned_addr learned my addr 172.21.15.28:0/931792095 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05d019fb00 con 0x55ef99c349a0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05d019f430 con 0x55ef99c352e0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00731d0 con 0x55ef99c3a370 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05d5472640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99cb3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.883+0000 7f05dac70740 10 client.0 _ll_register_callbacks cb 0x7f05cc00d7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 -- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 msgr2=0x55ef99cb3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99cb3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 -- 172.21.15.28:0/931792095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 msgr2=0x55ef99c437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef99cb40c0 con 0x55ef99c349a0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05d5472640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f05c00096c0 tx=0x7f05c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c572b914c7e25872 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0035050 con 0x55ef99c349a0 2024-09-19T09:42:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f05c0003710 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.884+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c00316b0 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f05c0003910 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 10 client.17037 fetch_fsmap learned FSMap version 47 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ef99cb5390 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c00316b0 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f05c003a040 con 0x55ef99c349a0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 10 client.17037 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 20 client.17037 populate_metadata read hostname 'smithi028' 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99cb57d0 0x55ef99cb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 4 client.17037 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef99b9fd70 con 0x55ef99cb57d0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.885+0000 7f05d5c73640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99cb57d0 0x55ef99cb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.886+0000 7f05b97fa640 10 client.17037 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.886+0000 7f05d5c73640 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99cb57d0 0x55ef99cb7bc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63de2cde0f3040db server_cookie=8251c60f03ff3e8e in_seq=0 out_seq=0 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05b97fa640 1 -- 172.21.15.28:0/931792095 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef99b9fd70 con 0x55ef99cb57d0 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05b97fa640 10 client.17037 handle_command_reply: tid=0 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 client.17037 shutdown 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 msgr2=0x55ef99c43270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f05c00096c0 tx=0x7f05c002a410 comp rx=0 tx=0).stop 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 shutdown_connections 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef99c352e0 0x55ef99c437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55ef99cb57d0 0x55ef99cb7bc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef99c3a370 0x55ef99cb3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 --2- 172.21.15.28:0/931792095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ef99c349a0 0x55ef99c43270 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05b97fa640 0 client.17037 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.968+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 >> 172.21.15.28:0/931792095 conn(0x55ef99c30010 msgr2=0x55ef99c2e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.969+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 shutdown_connections 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.969+0000 7f05dac70740 1 -- 172.21.15.28:0/931792095 wait complete. 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.969+0000 7f05dac70740 20 client.17037 trim_cache size 0 max 16384 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:{ 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "message": "", 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout: "return_code": 0 2024-09-19T09:42:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stdout:} 2024-09-19T09:42:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.969+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 msgr2=0x7f05bc0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.969+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 0x7f05bc0a5850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f05c8003bf0 tx=0x7f05c8003ac0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 msgr2=0x7f05d019d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05c8003fd0 tx=0x7f05c800fcc0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 shutdown_connections 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f05bc0a3390 0x7f05bc0a5850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05d0102620 0x7f05d019d260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05d0101c60 0x7f05d0196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 --2- 172.21.15.28:0/713047380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05d0100a70 0x7f05d0195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 >> 172.21.15.28:0/713047380 conn(0x7f05d00fc1f0 msgr2=0x7f05d0105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 shutdown_connections 2024-09-19T09:42:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi028.front.sepia.ceph.com.smithi028.stderr:2024-09-19T09:42:03.970+0000 7f05d7ed7640 1 -- 172.21.15.28:0/713047380 wait complete. 2024-09-19T09:42:04.998 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:04 smithi028 ceph-mon[30972]: pgmap v622: 97 pgs: 97 active+clean; 100 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.5 MiB/s wr, 499 op/s 2024-09-19T09:42:04.998 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:04 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-09-19T09:42:04.998 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:04 smithi028 systemd[1]: Stopped Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:04.998 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:04 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service: Consumed 8.595s CPU time. 2024-09-19T09:42:04.998 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:04 smithi028 systemd[1]: Starting Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:05.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:04 smithi078 ceph-mon[35619]: pgmap v622: 97 pgs: 97 active+clean; 100 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.5 MiB/s wr, 499 op/s 2024-09-19T09:42:05.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:04 smithi038 ceph-mon[35271]: pgmap v622: 97 pgs: 97 active+clean; 100 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.5 MiB/s wr, 499 op/s 2024-09-19T09:42:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:04 smithi028 podman[100577]: 2024-09-19 09:42:04.90165657 +0000 UTC m=+0.024502616 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:42:05.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 podman[100577]: 2024-09-19 09:42:05.21320885 +0000 UTC m=+0.336054920 container create 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:42:05.678 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 podman[100577]: 2024-09-19 09:42:05.432410649 +0000 UTC m=+0.555256683 container init 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T09:42:05.678 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 podman[100577]: 2024-09-19 09:42:05.435644682 +0000 UTC m=+0.558490717 container start 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:42:05.679 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 ceph-mds[100597]: -- 172.21.15.28:0/1349111592 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7be44f860 con 0x55f7be42d400 2024-09-19T09:42:05.679 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[100591]: starting mds.d at 2024-09-19T09:42:05.679 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 bash[100577]: 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 2024-09-19T09:42:05.679 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:42:05 smithi028 systemd[1]: Started Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:06.008 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:05 smithi078 ceph-mon[35619]: daemon mds.d restarted 2024-09-19T09:42:06.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:05 smithi028 ceph-mon[30972]: daemon mds.d restarted 2024-09-19T09:42:06.077 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:05 smithi038 ceph-mon[35271]: daemon mds.d restarted 2024-09-19T09:42:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: pgmap v623: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-09-19T09:42:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:42:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6836/1897083288,v1:172.21.15.28:6837/1897083288] up:boot 2024-09-19T09:42:07.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:42:07.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:42:07.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:06 smithi028 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:42:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: pgmap v623: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-09-19T09:42:07.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:42:07.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6836/1897083288,v1:172.21.15.28:6837/1897083288] up:boot 2024-09-19T09:42:07.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:42:07.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:42:07.108 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:06 smithi078 ceph-mon[35619]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: pgmap v623: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 423 op/s 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6836/1897083288,v1:172.21.15.28:6837/1897083288] up:boot 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:42:07.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:06 smithi038 ceph-mon[35271]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:42:08.739 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-19T09:42:08.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T09:42:09.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:08 smithi028 ceph-mon[30972]: pgmap v625: 97 pgs: 97 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-19T09:42:09.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:08 smithi078 ceph-mon[35619]: pgmap v625: 97 pgs: 97 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-19T09:42:09.164 DEBUG:teuthology.parallel:result is None 2024-09-19T09:42:09.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:42:09.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:08 smithi038 ceph-mon[35271]: pgmap v625: 97 pgs: 97 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-19T09:42:09.204 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:42:09.204 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T09:42:09.219 INFO:tasks.quiescer:joining Quiescers 2024-09-19T09:42:09.219 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T09:42:09.219 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:42:09.501 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.956+0000 7f3507af6640 1 -- 172.21.15.28:0/3001265074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500101de0 msgr2=0x7f35001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.956+0000 7f3507af6640 1 --2- 172.21.15.28:0/3001265074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500101de0 0x7f35001021c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009900 tx=0x7f34fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 -- 172.21.15.28:0/3001265074 shutdown_connections 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 --2- 172.21.15.28:0/3001265074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500106f10 0x7f350010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 --2- 172.21.15.28:0/3001265074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 0x7f35001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 --2- 172.21.15.28:0/3001265074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500101de0 0x7f35001021c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 -- 172.21.15.28:0/3001265074 >> 172.21.15.28:0/3001265074 conn(0x7f35000fd8c0 msgr2=0x7f35000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 -- 172.21.15.28:0/3001265074 shutdown_connections 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 -- 172.21.15.28:0/3001265074 wait complete. 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 Processor -- start 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.957+0000 7f3507af6640 1 -- start start 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 0x7f3500198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3500111f70 con 0x7f3500106f10 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3500111df0 con 0x7f3500101de0 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3507af6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35001120f0 con 0x7f3500102790 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3506093640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.28:49854/0 (socket says 172.21.15.28:49854) 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3506093640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37884/0 (socket says 172.21.15.28:37884) 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 -- 172.21.15.28:0/4173061075 learned_addr learned my addr 172.21.15.28:0/4173061075 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505091640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 0x7f3500198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 -- 172.21.15.28:0/4173061075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 msgr2=0x7f3500198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 0x7f3500198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 -- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 msgr2=0x7f350019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 -- 172.21.15.28:0/4173061075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350019d400 con 0x7f3500101de0 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3506093640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.958+0000 7f3505892640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006010 tx=0x7f34fc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:09.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.959+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc03f030 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.959+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350019d690 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.959+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34fc003ce0 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.959+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc049060 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.959+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35001a4fd0 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.960+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35001021c0 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.961+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f34fc042030 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.962+0000 7f34f6ffd640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 0x7f34ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.962+0000 7f3505091640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 0x7f34ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.962+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f34fc0f0220 con 0x7f3500101de0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.962+0000 7f3505091640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 0x7f34ec0a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f35000fdce0 tx=0x7f34f000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:09.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:09.966+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34fc0bc920 con 0x7f3500101de0 2024-09-19T09:42:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.182+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f350010a150 con 0x7f3500101de0 2024-09-19T09:42:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.183+0000 7f34f6ffd640 1 -- 172.21.15.28:0/4173061075 <== mon.2 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11965 (secure 0 0 0) 0x7f34fc0c0c20 con 0x7f3500101de0 2024-09-19T09:42:10.186 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:10.186 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":49,"btime":"2024-09-19T09:42:05:685861+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":17043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/1897083288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":1897083288},{"type":"v1","addr":"172.21.15.28:6837","nonce":1897083288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:42:05.685838+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26392":{"gid":26392,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2606259516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2606259516},{"type":"v1","addr":"172.21.15.78:6833","nonce":2606259516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:42:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 49 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 msgr2=0x7f34ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 0x7f34ec0a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f35000fdce0 tx=0x7f34f000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 msgr2=0x7f3500198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006010 tx=0x7f34fc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 shutdown_connections 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f34ec0a3240 0x7f34ec0a5700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3500106f10 0x7f350019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3500102790 0x7f3500198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 --2- 172.21.15.28:0/4173061075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3500101de0 0x7f3500198450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 >> 172.21.15.28:0/4173061075 conn(0x7f35000fd8c0 msgr2=0x7f35000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 shutdown_connections 2024-09-19T09:42:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:10.186+0000 7f3507af6640 1 -- 172.21.15.28:0/4173061075 wait complete. 2024-09-19T09:42:10.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24499 quiesce db --cancel --all' 2024-09-19T09:42:10.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24499 quiesce db --cancel --all 2024-09-19T09:42:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:10 smithi028 ceph-mon[30972]: pgmap v626: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-19T09:42:11.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:10 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4173061075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:11.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:10 smithi078 ceph-mon[35619]: pgmap v626: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-19T09:42:11.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:10 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4173061075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:10 smithi038 ceph-mon[35271]: pgmap v626: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-19T09:42:11.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:10 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4173061075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:11.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 -- 172.21.15.28:0/374987444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 msgr2=0x7f16d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 --2- 172.21.15.28:0/374987444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d81067e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 -- 172.21.15.28:0/374987444 shutdown_connections 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 --2- 172.21.15.28:0/374987444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8106f10 0x7f16d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 --2- 172.21.15.28:0/374987444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d81067e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 --2- 172.21.15.28:0/374987444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8101de0 0x7f16d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 -- 172.21.15.28:0/374987444 >> 172.21.15.28:0/374987444 conn(0x7f16d80fd8c0 msgr2=0x7f16d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 -- 172.21.15.28:0/374987444 shutdown_connections 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.593+0000 7f16de3ad640 1 -- 172.21.15.28:0/374987444 wait complete. 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 Processor -- start 2024-09-19T09:42:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 -- start start 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8106f10 0x7f16d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8111f70 con 0x7f16d8102790 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8111df0 con 0x7f16d8106f10 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16de3ad640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81120f0 con 0x7f16d8101de0 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:44450/0 (socket says 172.21.15.28:44450) 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.594+0000 7f16d7fff640 1 -- 172.21.15.28:0/3526735262 learned_addr learned my addr 172.21.15.28:0/3526735262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8106f10 0x7f16d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 -- 172.21.15.28:0/3526735262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 msgr2=0x7f16d80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 -- 172.21.15.28:0/3526735262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8106f10 msgr2=0x7f16d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8106f10 0x7f16d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 -- 172.21.15.28:0/3526735262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81a6900 con 0x7f16d8102790 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16d7fff640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16cffff640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d8074a30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005940 tx=0x7f16c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16d5ffb640 1 -- 172.21.15.28:0/3526735262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c003f030 con 0x7f16d8102790 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16d5ffb640 1 -- 172.21.15.28:0/3526735262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c0003ce0 con 0x7f16d8102790 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81a6b90 con 0x7f16d8102790 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16d5ffb640 1 -- 172.21.15.28:0/3526735262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0035070 con 0x7f16d8102790 2024-09-19T09:42:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.595+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a7070 con 0x7f16d8102790 2024-09-19T09:42:11.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16d5ffb640 1 -- 172.21.15.28:0/3526735262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f16c0042030 con 0x7f16d8102790 2024-09-19T09:42:11.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 Processor -- start 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 -- start start 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 0x55abdb2dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55abdb233200 con 0x55abdb2cea00 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abdb1d9dc0 con 0x55abdb2d9be0 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16e0786740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abdb22f1c0 con 0x55abdb2cf320 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16dc94a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16dc94a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37916/0 (socket says 172.21.15.28:37916) 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16dc94a640 1 -- 172.21.15.28:0/2291558383 learned_addr learned my addr 172.21.15.28:0/2291558383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.598+0000 7f16d7fff640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16cffff640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 0x55abdb2dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16d5ffb640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 0x7f16c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 -- 172.21.15.28:0/2291558383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 msgr2=0x55abdb2d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 -- 172.21.15.28:0/2291558383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 msgr2=0x55abdb2dbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 0x55abdb2dbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 -- 172.21.15.28:0/2291558383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abdafa4620 con 0x55abdb2cea00 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16d7fff640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 0x7f16c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16cffff640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 0x55abdb2dbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16d5ffb640 1 -- 172.21.15.28:0/3526735262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f16c00efcc0 con 0x7f16d8102790 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16dc94a640 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f16d0010440 tx=0x7f16d0010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8511ba8a0996ddee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16ce7fc640 1 -- 172.21.15.28:0/2291558383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d910 con 0x55abdb2cea00 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16ce7fc640 1 -- 172.21.15.28:0/2291558383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d001c440 con 0x55abdb2cea00 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.599+0000 7f16ce7fc640 1 -- 172.21.15.28:0/2291558383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d001b630 con 0x55abdb2cea00 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 -- 172.21.15.28:0/2291558383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 msgr2=0x55abdb2cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 0x7f16c40a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16d0021000 tx=0x7f16d001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f16d0010440 tx=0x7f16d0010840 comp rx=0 tx=0).stop 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 -- 172.21.15.28:0/2291558383 shutdown_connections 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2d9be0 0x55abdb2dbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 --2- 172.21.15.28:0/2291558383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cea00 0x55abdb2cede0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 -- 172.21.15.28:0/2291558383 >> 172.21.15.28:0/2291558383 conn(0x55abdb2caf90 msgr2=0x55abdb2cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 -- 172.21.15.28:0/2291558383 shutdown_connections 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.600+0000 7f16e0786740 1 -- 172.21.15.28:0/2291558383 wait complete. 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 -- start start 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 Processor -- start 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55abdb2dc7b0 con 0x55abdb2d9be0 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abdb1cdc50 con 0x55abdb2cea00 2024-09-19T09:42:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abdb2dc930 con 0x55abdb2cf320 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16e0786740 10 client.0 _ll_register_callbacks cb 0x55abdb1dfba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16dc94a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16dc94a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37932/0 (socket says 172.21.15.28:37932) 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16dc94a640 1 -- 172.21.15.28:0/3730114342 learned_addr learned my addr 172.21.15.28:0/3730114342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16d7fff640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.601+0000 7f16cffff640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 -- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 msgr2=0x55abdb2d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 -- 172.21.15.28:0/3730114342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 msgr2=0x55abdb2d9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2d9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 -- 172.21.15.28:0/3730114342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abdb2d7480 con 0x55abdb2d9be0 2024-09-19T09:42:11.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16dc94a640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009640 tx=0x7f16d001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452a3e9f9c430626 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0036060 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f16d001d890 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d001c440 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f16d002d040 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16e0786740 10 client.17061 fetch_fsmap learned FSMap version 49 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.602+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55abdb2e82a0 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d001c440 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 49) ==== 8386+0+0 (secure 0 0 0) 0x7f16d000f040 con 0x55abdb2d9be0 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 10 client.17061 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 10 client.17061 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 20 client.17061 populate_metadata read hostname 'smithi028' 2024-09-19T09:42:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb2e96a0 0x55abdb2eba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 4 client.17061 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:42:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abdb240770 con 0x55abdb2e96a0 2024-09-19T09:42:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.603+0000 7f16d7fff640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb2e96a0 0x55abdb2eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.604+0000 7f16d7fff640 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb2e96a0 0x55abdb2eba90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db6b0f7935e6fd2 server_cookie=1217292094bdd91a in_seq=0 out_seq=0 2024-09-19T09:42:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.604+0000 7f16b77fe640 10 client.17061 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.604+0000 7f16b77fe640 1 -- 172.21.15.28:0/3730114342 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abdb240770 con 0x55abdb2e96a0 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.604+0000 7f16b77fe640 10 client.17061 handle_command_reply: tid=0 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.604+0000 7f16e0786740 1 client.17061 shutdown 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 msgr2=0x55abdb2d6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009640 tx=0x7f16d001c880 comp rx=0 tx=0).stop 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 shutdown_connections 2024-09-19T09:42:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb2e96a0 0x55abdb2eba90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2d9be0 0x55abdb2d6c80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2cf320 0x55abdb2d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 --2- 172.21.15.28:0/3730114342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2d9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 >> 172.21.15.28:0/3730114342 conn(0x55abdb2caf90 msgr2=0x55abdb2ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16b77fe640 0 client.17061 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 shutdown_connections 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 1 -- 172.21.15.28:0/3730114342 wait complete. 2024-09-19T09:42:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.605+0000 7f16e0786740 20 client.17061 trim_cache size 0 max 16384 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 Processor -- start 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 -- start start 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb2f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81af6b0 con 0x55abdb2cf320 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b12e0 con 0x55abdb2cea00 2024-09-19T09:42:11.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16e0786740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81afe10 con 0x55abdb2d9be0 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.622+0000 7f16cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37942/0 (socket says 172.21.15.28:37942) 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 -- 172.21.15.28:0/1931802889 learned_addr learned my addr 172.21.15.28:0/1931802889 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16d7fff640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16dc94a640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb2f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 -- 172.21.15.28:0/1931802889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 msgr2=0x55abdb2f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb2f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 -- 172.21.15.28:0/1931802889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 msgr2=0x55abdb2e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 -- 172.21.15.28:0/1931802889 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abdb2cf150 con 0x55abdb2cf320 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16d7fff640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cffff640 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16c00bc350 tx=0x7f16c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5297b55681a709d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cdffb640 1 -- 172.21.15.28:0/1931802889 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0049050 con 0x55abdb2cf320 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cdffb640 1 -- 172.21.15.28:0/1931802889 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c00bc3c0 con 0x55abdb2cf320 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.623+0000 7f16cdffb640 1 -- 172.21.15.28:0/1931802889 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c00f1070 con 0x55abdb2cf320 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- 172.21.15.28:0/1931802889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 msgr2=0x55abdb2f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16c00bc350 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- 172.21.15.28:0/1931802889 shutdown_connections 2024-09-19T09:42:11.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb2f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2f1200 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 --2- 172.21.15.28:0/1931802889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- 172.21.15.28:0/1931802889 >> 172.21.15.28:0/1931802889 conn(0x55abdb2caf90 msgr2=0x55abdb2d5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- 172.21.15.28:0/1931802889 shutdown_connections 2024-09-19T09:42:11.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- 172.21.15.28:0/1931802889 wait complete. 2024-09-19T09:42:11.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 -- start start 2024-09-19T09:42:11.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.624+0000 7f16e0786740 1 Processor -- start 2024-09-19T09:42:11.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb34c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b0c10 con 0x55abdb2cf320 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b0540 con 0x55abdb2cea00 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81ad690 con 0x55abdb2d9be0 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16e0786740 10 client.0 _ll_register_callbacks cb 0x7f16d0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37952/0 (socket says 172.21.15.28:37952) 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 -- 172.21.15.28:0/1774914736 learned_addr learned my addr 172.21.15.28:0/1774914736 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16d7fff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16dc94a640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb34c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 -- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 msgr2=0x55abdb34c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb34c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 -- 172.21.15.28:0/1774914736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 msgr2=0x55abdb2df250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2df250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 -- 172.21.15.28:0/1774914736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abdb34c8b0 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16dc94a640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb34c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16cffff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f16c00b8d90 tx=0x7f16c0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ba61613fc002f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.625+0000 7f16b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c00da060 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f16c00ba460 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c00bc3c0 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f16c0049020 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 10 client.17067 fetch_fsmap learned FSMap version 49 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55abdb353c20 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c00bc3c0 con 0x55abdb2cf320 2024-09-19T09:42:11.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 49) ==== 8386+0+0 (secure 0 0 0) 0x7f16c00c4af0 con 0x55abdb2cf320 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 10 client.17067 fetch_fsmap finished waiting for FSMap version 49 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 10 client.17067 resolve_mds: validated gid 24499 aka daemon mds.i 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 20 client.17067 populate_metadata read hostname 'smithi028' 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb3541c0 0x55abdb3565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 4 client.17067 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 --> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55abdb23ac80 con 0x55abdb3541c0 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.626+0000 7f16d7fff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb3541c0 0x55abdb3565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16d7fff640 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb3541c0 0x55abdb3565e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76da20ac79c77985 server_cookie=ab1dce63b73d4ff9 in_seq=0 out_seq=0 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16b77fe640 10 client.17067 ms_handle_connect on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:11.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16b77fe640 1 -- 172.21.15.28:0/1774914736 <== mds.0 v2:172.21.15.78:6838/3581897338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55abdb23ac80 con 0x55abdb3541c0 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16b77fe640 10 client.17067 handle_command_reply: tid=0 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16e0786740 1 client.17067 shutdown 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 msgr2=0x55abdb2df790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.627+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f16c00b8d90 tx=0x7f16c0009680 comp rx=0 tx=0).stop 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 shutdown_connections 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:6838/3581897338,v1:172.21.15.78:6839/3581897338] conn(0x55abdb3541c0 0x55abdb3565e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abdb2d9be0 0x55abdb34c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55abdb2cf320 0x55abdb2df790 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16b77fe640 0 client.17067 ms_handle_reset on v2:172.21.15.78:6838/3581897338 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 --2- 172.21.15.28:0/1774914736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abdb2cea00 0x55abdb2df250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 >> 172.21.15.28:0/1774914736 conn(0x55abdb2caf90 msgr2=0x55abdb2c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 shutdown_connections 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 1 -- 172.21.15.28:0/1774914736 wait complete. 2024-09-19T09:42:11.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16e0786740 20 client.17067 trim_cache size 0 max 16384 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 49, 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout: "leader": 24499, 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout: "set_version": 32, 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout: "sets": {} 2024-09-19T09:42:11.637 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 msgr2=0x7f16c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 0x7f16c40a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16d0021000 tx=0x7f16d001e000 comp rx=0 tx=0).stop 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 msgr2=0x7f16d8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d8074a30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005940 tx=0x7f16c00042f0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 shutdown_connections 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f16c40a3240 0x7f16c40a5700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16d8106f10 0x7f16d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16d8102790 0x7f16d8074a30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 --2- 172.21.15.28:0/3526735262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d8101de0 0x7f16d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 >> 172.21.15.28:0/3526735262 conn(0x7f16d80fd8c0 msgr2=0x7f16d80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.628+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 shutdown_connections 2024-09-19T09:42:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:11.629+0000 7f16de3ad640 1 -- 172.21.15.28:0/3526735262 wait complete. 2024-09-19T09:42:12.274 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T09:42:12.275 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T09:42:12.285 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:42:12.294 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T09:42:12.304 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T09:42:12.304 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-09-19T09:42:12.305 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T09:42:12.315 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:42:12.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 -- 172.21.15.28:0/4151005405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4101c30 msgr2=0x7efdc41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 --2- 172.21.15.28:0/4151005405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4101c30 0x7efdc41020b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009900 tx=0x7efdb402a410 comp rx=0 tx=0).stop 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 -- 172.21.15.28:0/4151005405 shutdown_connections 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 --2- 172.21.15.28:0/4151005405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc41025f0 0x7efdc4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 --2- 172.21.15.28:0/4151005405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc4101c30 0x7efdc41020b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 --2- 172.21.15.28:0/4151005405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 -- 172.21.15.28:0/4151005405 >> 172.21.15.28:0/4151005405 conn(0x7efdc40fc1a0 msgr2=0x7efdc40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 -- 172.21.15.28:0/4151005405 shutdown_connections 2024-09-19T09:42:12.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.438+0000 7efdcb077640 1 -- 172.21.15.28:0/4151005405 wait complete. 2024-09-19T09:42:12.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 Processor -- start 2024-09-19T09:42:12.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 -- start start 2024-09-19T09:42:12.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4101c30 0x7efdc419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 0x7efdc419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410c6c0 con 0x7efdc41025f0 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410c540 con 0x7efdc4101c30 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdcb077640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410c840 con 0x7efdc4100a40 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc8e13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc9614640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 0x7efdc419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4101c30 0x7efdc419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc8e13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.28:44502/0 (socket says 172.21.15.28:44502) 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc8e13640 1 -- 172.21.15.28:0/930948801 learned_addr learned my addr 172.21.15.28:0/930948801 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.439+0000 7efdc9614640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 0x7efdc419f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37966/0 (socket says 172.21.15.28:37966) 2024-09-19T09:42:12.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 -- 172.21.15.28:0/930948801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4101c30 msgr2=0x7efdc419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4101c30 0x7efdc419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 -- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 msgr2=0x7efdc419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 0x7efdc419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 -- 172.21.15.28:0/930948801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc419fad0 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc8e13640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9c0 tx=0x7efdac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00cd80 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc419fd60 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdac0035f0 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac010690 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.440+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc41a01d0 con 0x7efdc4100a40 2024-09-19T09:42:12.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.441+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc4107e80 con 0x7efdc4100a40 2024-09-19T09:42:12.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.442+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7efdac0037a0 con 0x7efdc4100a40 2024-09-19T09:42:12.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.443+0000 7efdc1ffb640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 0x7efdb00a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.443+0000 7efdc3fff640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 0x7efdb00a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.443+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7efdac0cfc90 con 0x7efdc4100a40 2024-09-19T09:42:12.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.444+0000 7efdc3fff640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 0x7efdb00a5710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efdb40314e0 tx=0x7efdb4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:12.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.447+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efdac09c980 con 0x7efdc4100a40 2024-09-19T09:42:12.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.670+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdc4062ef0 con 0x7efdc4100a40 2024-09-19T09:42:12.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.752+0000 7efdc1ffb640 1 -- 172.21.15.28:0/930948801 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22580 (secure 0 0 0) 0x7efdac09c280 con 0x7efdc4100a40 2024-09-19T09:42:12.754 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:12.754 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":85,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:42:05.465135+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 msgr2=0x7efdb00a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 0x7efdb00a5710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efdb40314e0 tx=0x7efdb4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 msgr2=0x7efdc406ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9c0 tx=0x7efdac00ee80 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 shutdown_connections 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7efdb00a3250 0x7efdb00a5710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdc4101c30 0x7efdc419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc41025f0 0x7efdc419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 --2- 172.21.15.28:0/930948801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdc4100a40 0x7efdc406ae80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.754+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 >> 172.21.15.28:0/930948801 conn(0x7efdc40fc1a0 msgr2=0x7efdc4105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.755+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 shutdown_connections 2024-09-19T09:42:12.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.755+0000 7efdcb077640 1 -- 172.21.15.28:0/930948801 wait complete. 2024-09-19T09:42:12.769 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:26:05.815869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:30:39.464809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:30:41.429176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:42:12.769 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:42:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- 172.21.15.28:0/3439886459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48104f90 msgr2=0x7f3f48107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3439886459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48104f90 0x7f3f48107380 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009900 tx=0x7f3f3c02a410 comp rx=0 tx=0).stop 2024-09-19T09:42:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- 172.21.15.28:0/3439886459 shutdown_connections 2024-09-19T09:42:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3439886459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f481078c0 0x7f3f48109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3439886459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48104f90 0x7f3f48107380 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3439886459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f48102660 0x7f3f48104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- 172.21.15.28:0/3439886459 >> 172.21.15.28:0/3439886459 conn(0x7f3f480fc180 msgr2=0x7f3f480fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- 172.21.15.28:0/3439886459 shutdown_connections 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- 172.21.15.28:0/3439886459 wait complete. 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 Processor -- start 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- start start 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 0x7f3f481984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f481078c0 0x7f3f4819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d780 con 0x7f3f48102660 2024-09-19T09:42:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.875+0000 7f3f506f1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d600 con 0x7f3f48104f90 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f506f1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d900 con 0x7f3f481078c0 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37998/0 (socket says 172.21.15.28:37998) 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 -- 172.21.15.28:0/3507689110 learned_addr learned my addr 172.21.15.28:0/3507689110 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4ec8e640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f481078c0 0x7f3f4819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4dc8c640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 0x7f3f481984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 -- 172.21.15.28:0/3507689110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f481078c0 msgr2=0x7f3f4819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f481078c0 0x7f3f4819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 -- 172.21.15.28:0/3507689110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 msgr2=0x7f3f481984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 0x7f3f481984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 -- 172.21.15.28:0/3507689110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4819fb90 con 0x7f3f48102660 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4dc8c640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 0x7f3f481984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f4e48d640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d950 tx=0x7f3f3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f48102660 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f38009c30 con 0x7f3f48102660 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800f6b0 con 0x7f3f48102660 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4819fe80 con 0x7f3f48102660 2024-09-19T09:42:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.876+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f481a0290 con 0x7f3f48102660 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.877+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480fdc70 con 0x7f3f48102660 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.879+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3f380109d0 con 0x7f3f48102660 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.880+0000 7f3f477fe640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 0x7f3f300a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.880+0000 7f3f4dc8c640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 0x7f3f300a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.880+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f3f380cf620 con 0x7f3f48102660 2024-09-19T09:42:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.880+0000 7f3f4dc8c640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 0x7f3f300a5710 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:12.883+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f38009de0 con 0x7f3f48102660 2024-09-19T09:42:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:12 smithi028 ceph-mon[30972]: pgmap v627: 97 pgs: 97 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-19T09:42:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:42:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:42:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:42:13.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:12 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:42:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.059+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f4802da30 con 0x7f3f48102660 2024-09-19T09:42:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.060+0000 7f3f477fe640 1 -- 172.21.15.28:0/3507689110 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f3f3809c310 con 0x7f3f48102660 2024-09-19T09:42:13.061 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-19T09:42:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 msgr2=0x7f3f300a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 0x7f3f300a5710 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c031400 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 msgr2=0x7f3f48197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d950 tx=0x7f3f3800de10 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 shutdown_connections 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f3f300a3250 0x7f3f300a5710 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f48104f90 0x7f3f481984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f481078c0 0x7f3f4819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 --2- 172.21.15.28:0/3507689110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f48102660 0x7f3f48197f70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 >> 172.21.15.28:0/3507689110 conn(0x7f3f480fc180 msgr2=0x7f3f48104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 shutdown_connections 2024-09-19T09:42:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.062+0000 7f3f506f1640 1 -- 172.21.15.28:0/3507689110 wait complete. 2024-09-19T09:42:13.074 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:42:13.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:12 smithi078 ceph-mon[35619]: pgmap v627: 97 pgs: 97 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-19T09:42:13.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:42:13.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:42:13.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:42:13.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:12 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:42:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:12 smithi038 ceph-mon[35271]: pgmap v627: 97 pgs: 97 active+clean; 48 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.6 MiB/s wr, 408 op/s 2024-09-19T09:42:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:42:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:42:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:42:13.176 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:12 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' 2024-09-19T09:42:13.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 -- 172.21.15.28:0/450330703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b81025d0 msgr2=0x7f70b81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/450330703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b81025d0 0x7f70b81049c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009900 tx=0x7f70a802a410 comp rx=0 tx=0).stop 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 -- 172.21.15.28:0/450330703 shutdown_connections 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/450330703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8104f00 0x7f70b81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/450330703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b8107830 0x7f70b8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/450330703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b81025d0 0x7f70b81049c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 -- 172.21.15.28:0/450330703 >> 172.21.15.28:0/450330703 conn(0x7f70b80fc190 msgr2=0x7f70b80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 -- 172.21.15.28:0/450330703 shutdown_connections 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.180+0000 7f70bdb4b640 1 -- 172.21.15.28:0/450330703 wait complete. 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 Processor -- start 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 -- start start 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b81025d0 0x7f70b8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8107830 0x7f70b819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d840 con 0x7f70b8104f00 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d6c0 con 0x7f70b8107830 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70bdb4b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d9c0 con 0x7f70b81025d0 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38018/0 (socket says 172.21.15.28:38018) 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 -- 172.21.15.28:0/336304980 learned_addr learned my addr 172.21.15.28:0/336304980 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b77fe640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b81025d0 0x7f70b8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b7fff640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8107830 0x7f70b819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 -- 172.21.15.28:0/336304980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b81025d0 msgr2=0x7f70b8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b81025d0 0x7f70b8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 -- 172.21.15.28:0/336304980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8107830 msgr2=0x7f70b819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8107830 0x7f70b819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.181+0000 7f70b6ffd640 1 -- 172.21.15.28:0/336304980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b819fb70 con 0x7f70b8104f00 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70b6ffd640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f70a000d770 tx=0x7f70a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0017070 con 0x7f70b8104f00 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0009c30 con 0x7f70b8104f00 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b819fe60 con 0x7f70b8104f00 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a000f620 con 0x7f70b8104f00 2024-09-19T09:42:13.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.182+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81a0320 con 0x7f70b8104f00 2024-09-19T09:42:13.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.183+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b80fdc80 con 0x7f70b8104f00 2024-09-19T09:42:13.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.184+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f70a001a070 con 0x7f70b8104f00 2024-09-19T09:42:13.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.185+0000 7f70b4ff9640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 0x7f70a40a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.185+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f70a00cf320 con 0x7f70b8104f00 2024-09-19T09:42:13.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.185+0000 7f70b77fe640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 0x7f70a40a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.186+0000 7f70b77fe640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 0x7f70a40a5710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.188+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70a0009de0 con 0x7f70b8104f00 2024-09-19T09:42:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.366+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70b802da10 con 0x7f70b8104f00 2024-09-19T09:42:13.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.367+0000 7f70b4ff9640 1 -- 172.21.15.28:0/336304980 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f70a009b9c0 con 0x7f70b8104f00 2024-09-19T09:42:13.368 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.368+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 msgr2=0x7f70a40a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.368+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 0x7f70a40a5710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70a8000c00 tx=0x7f70a80315b0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.368+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 msgr2=0x7f70b8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.368+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f70a000d770 tx=0x7f70a000dc30 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 shutdown_connections 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f70a40a3250 0x7f70a40a5710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b8107830 0x7f70b819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70b8104f00 0x7f70b8198490 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 --2- 172.21.15.28:0/336304980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b81025d0 0x7f70b8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 >> 172.21.15.28:0/336304980 conn(0x7f70b80fc190 msgr2=0x7f70b8103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 shutdown_connections 2024-09-19T09:42:13.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.369+0000 7f70bdb4b640 1 -- 172.21.15.28:0/336304980 wait complete. 2024-09-19T09:42:13.381 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:42:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 -- 172.21.15.28:0/2810354317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 msgr2=0x7ff4801031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 --2- 172.21.15.28:0/2810354317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801031a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff468009900 tx=0x7ff46802a410 comp rx=0 tx=0).stop 2024-09-19T09:42:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 -- 172.21.15.28:0/2810354317 shutdown_connections 2024-09-19T09:42:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 --2- 172.21.15.28:0/2810354317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 0x7ff480109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 --2- 172.21.15.28:0/2810354317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801031a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 --2- 172.21.15.28:0/2810354317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 0x7ff480101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 -- 172.21.15.28:0/2810354317 >> 172.21.15.28:0/2810354317 conn(0x7ff4800fd270 msgr2=0x7ff4800ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 -- 172.21.15.28:0/2810354317 shutdown_connections 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.489+0000 7ff4855e4640 1 -- 172.21.15.28:0/2810354317 wait complete. 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 Processor -- start 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 -- start start 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 0x7ff480197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 0x7ff48019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff48010d7b0 con 0x7ff480102d20 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff48010d630 con 0x7ff4801036e0 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff4855e4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff48010d930 con 0x7ff480101b30 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59046/0 (socket says 172.21.15.28:59046) 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 -- 172.21.15.28:0/4058207861 learned_addr learned my addr 172.21.15.28:0/4058207861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff47effd640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 0x7ff480197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff47f7fe640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 0x7ff48019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 -- 172.21.15.28:0/4058207861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 msgr2=0x7ff480197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 0x7ff480197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 -- 172.21.15.28:0/4058207861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 msgr2=0x7ff48019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 0x7ff48019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.490+0000 7ff477fff640 1 -- 172.21.15.28:0/4058207861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48019fab0 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff477fff640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff468009230 tx=0x7ff468031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46803f030 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff468003ad0 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48019fd40 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468035070 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.491+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801a01d0 con 0x7ff480102d20 2024-09-19T09:42:13.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.492+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff480108f60 con 0x7ff480102d20 2024-09-19T09:42:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.493+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff468042030 con 0x7ff480102d20 2024-09-19T09:42:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.494+0000 7ff47cff9640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 0x7ff46c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.494+0000 7ff47effd640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 0x7ff46c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.494+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7ff4680efe50 con 0x7ff480102d20 2024-09-19T09:42:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.494+0000 7ff47effd640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 0x7ff46c0a5710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff480102b80 tx=0x7ff470005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.498+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4680bc4f0 con 0x7ff480102d20 2024-09-19T09:42:13.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.674+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff480062e90 con 0x7ff480102d20 2024-09-19T09:42:13.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.674+0000 7ff47cff9640 1 -- 172.21.15.28:0/4058207861 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7ff4680c07f0 con 0x7ff480102d20 2024-09-19T09:42:13.676 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-09-19T09:42:13.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 msgr2=0x7ff46c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 0x7ff46c0a5710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff480102b80 tx=0x7ff470005f90 comp rx=0 tx=0).stop 2024-09-19T09:42:13.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 msgr2=0x7ff4801983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff468009230 tx=0x7ff468031b60 comp rx=0 tx=0).stop 2024-09-19T09:42:13.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 shutdown_connections 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7ff46c0a3250 0x7ff46c0a5710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4801036e0 0x7ff48019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff480102d20 0x7ff4801983d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 --2- 172.21.15.28:0/4058207861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff480101b30 0x7ff480197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 >> 172.21.15.28:0/4058207861 conn(0x7ff4800fd270 msgr2=0x7ff480106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 shutdown_connections 2024-09-19T09:42:13.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.677+0000 7ff4855e4640 1 -- 172.21.15.28:0/4058207861 wait complete. 2024-09-19T09:42:13.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 -- 172.21.15.28:0/1107788171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 msgr2=0x7f9d7c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 --2- 172.21.15.28:0/1107788171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c101160 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009900 tx=0x7f9d6c02a410 comp rx=0 tx=0).stop 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 -- 172.21.15.28:0/1107788171 shutdown_connections 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 --2- 172.21.15.28:0/1107788171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 0x7f9d7c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 --2- 172.21.15.28:0/1107788171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c101160 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 --2- 172.21.15.28:0/1107788171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 -- 172.21.15.28:0/1107788171 >> 172.21.15.28:0/1107788171 conn(0x7f9d7c0fc1b0 msgr2=0x7f9d7c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 -- 172.21.15.28:0/1107788171 shutdown_connections 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.803+0000 7f9d83df6640 1 -- 172.21.15.28:0/1107788171 wait complete. 2024-09-19T09:42:13.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 Processor -- start 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 -- start start 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 0x7f9d7c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10f9c0 con 0x7f9d7c100d00 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10f840 con 0x7f9d7c1016a0 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d83df6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10fb40 con 0x7f9d7c1003c0 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59064/0 (socket says 172.21.15.28:59064) 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 -- 172.21.15.28:0/630988803 learned_addr learned my addr 172.21.15.28:0/630988803 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81b92640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 -- 172.21.15.28:0/630988803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 msgr2=0x7f9d7c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d82393640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 0x7f9d7c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 -- 172.21.15.28:0/630988803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 msgr2=0x7f9d7c1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 0x7f9d7c1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 -- 172.21.15.28:0/630988803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c1a1cc0 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81b92640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.804+0000 7f9d81391640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009230 tx=0x7f9d6c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.805+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c03f030 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.805+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d6c003ad0 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.805+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c1a1f50 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.805+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c035070 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.805+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a23b0 con 0x7f9d7c100d00 2024-09-19T09:42:13.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.806+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c10b240 con 0x7f9d7c100d00 2024-09-19T09:42:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.807+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9d6c042030 con 0x7f9d7c100d00 2024-09-19T09:42:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.807+0000 7f9d72ffd640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 0x7f9d680a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:13.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.808+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f9d6c0eff20 con 0x7f9d7c100d00 2024-09-19T09:42:13.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.808+0000 7f9d81b92640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 0x7f9d680a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:13.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.808+0000 7f9d81b92640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 0x7f9d680a5710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009c20 tx=0x7f9d78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:13.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:13.812+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d6c0bcc70 con 0x7f9d7c100d00 2024-09-19T09:42:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/930948801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:14.014 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3507689110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:42:14.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/336304980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:42:14.015 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:13 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4058207861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:42:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.036+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9d7c062e90 con 0x7f9d7c100d00 2024-09-19T09:42:14.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.044+0000 7f9d72ffd640 1 -- 172.21.15.28:0/630988803 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742334 (secure 0 0 0) 0x7f9d6c0dc020 con 0x7f9d7c100d00 2024-09-19T09:42:14.055 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:14.055 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 20, 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "active_gid": 14150, 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "active_name": "x", 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "active_addrs": { 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6800", 2024-09-19T09:42:14.056 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 147346946 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6801", 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 147346946 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "active_addr": "172.21.15.28:6801/147346946", 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "active_change": "2024-09-19T09:23:47.303458+0000", 2024-09-19T09:42:14.057 INFO:teuthology.orchestra.run.smithi028.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "available": true, 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "standbys": [ 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "gid": 24109, 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "z", 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "available_modules": [ 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "alerts", 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.058 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.059 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.060 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.061 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.062 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.063 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.064 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.065 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_destination": { 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_destination", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.067 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_from_name": { 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_from_name", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "Ceph", 2024-09-19T09:42:14.068 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email From: name", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_host": { 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_host", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.069 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP server", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.070 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_password": { 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_password", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Password to authenticate with", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.071 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_port": { 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_port", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "465", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.072 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP port", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_sender": { 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_sender", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.073 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_ssl": { 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_ssl", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.074 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.075 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_user": { 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_user", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "User to authenticate as", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.076 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "balancer", 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "active": { 2024-09-19T09:42:14.077 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "active", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_time": { 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_time", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0000", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.079 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_weekday": { 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_weekday", 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.080 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.081 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_max_iterations": { 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "25", 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.082 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "250", 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.083 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_metrics": { 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_metrics", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:42:14.084 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_step": { 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_step", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.5", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0.001", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "0.999", 2024-09-19T09:42:14.085 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "end_time": { 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_time", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2359", 2024-09-19T09:42:14.086 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "end_weekday": { 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_weekday", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.087 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.088 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.089 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.090 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.091 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.092 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.093 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.094 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "min_score": { 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_score", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.095 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.096 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "upmap", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-compat", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "none", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "read", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap-read" 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Balancer mode", 2024-09-19T09:42:14.097 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_ids": { 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_ids", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.098 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.099 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_deviation": { 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_deviation", 2024-09-19T09:42:14.100 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_optimizations": { 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.102 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm", 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.103 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_down_multiplier": { 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_down_multiplier", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_refresh_rate": { 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_refresh_rate", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "20", 2024-09-19T09:42:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_starting_port": { 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_starting_port", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.106 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4721", 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:42:14.107 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_ptrace": { 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_ptrace", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.108 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_interval": { 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_interval", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.109 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:42:14.110 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.7", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.111 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm_log_destination": { 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm_log_destination", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.112 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "file", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "file,syslog", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "syslog" 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: "cgroups_split": { 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cgroups_split", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.114 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "config_checks_enabled": { 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_checks_enabled", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.115 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "config_dashboard": { 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_dashboard", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.116 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_alertmanager": { 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_alertmanager", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.117 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_base": { 2024-09-19T09:42:14.118 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_base", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.119 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_elasticsearch": { 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "elasticsearch container image", 2024-09-19T09:42:14.120 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_grafana": { 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_grafana", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.121 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_haproxy": { 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_haproxy", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.122 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "HAproxy container image", 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_agent": { 2024-09-19T09:42:14.123 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_collector": { 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:42:14.125 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_query": { 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.126 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger query container image", 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.127 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_keepalived": { 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_keepalived", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.128 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Keepalived container image", 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_loki": { 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_loki", 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.129 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Loki container image", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_node_exporter": { 2024-09-19T09:42:14.130 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_node_exporter", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.131 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_nvmeof": { 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_nvmeof", 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.132 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Nvme-of container image", 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_prometheus": { 2024-09-19T09:42:14.133 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_prometheus", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.134 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_promtail": { 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_promtail", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Promtail container image", 2024-09-19T09:42:14.135 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_samba": { 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_samba", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.136 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_snmp_gateway": { 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.137 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "container_init": { 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_init", 2024-09-19T09:42:14.138 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.139 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "daemon_cache_timeout": { 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.140 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "900", 2024-09-19T09:42:14.141 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "default_registry": { 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_registry", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.142 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.143 INFO:teuthology.orchestra.run.smithi028.stdout: "device_cache_timeout": { 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_cache_timeout", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1800", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:42:14.144 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "device_enhanced_scan": { 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_enhanced_scan", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.145 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "facts_cache_timeout": { 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "facts_cache_timeout", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.146 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.147 INFO:teuthology.orchestra.run.smithi028.stdout: "grafana_dashboards_path": { 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.148 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "host_check_interval": { 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "host_check_interval", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.149 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "hw_monitoring": { 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hw_monitoring", 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.150 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "inventory_list_all": { 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "inventory_list_all", 2024-09-19T09:42:14.151 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.153 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "log_refresh_metadata": { 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_refresh_metadata", 2024-09-19T09:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.155 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.156 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.157 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.158 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.159 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.160 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:42:14.161 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "*", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.162 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "max_count_per_host": { 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_count_per_host", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.163 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd_draining_count": { 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_osd_draining_count", 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.164 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "migration_current": { 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "migration_current", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.165 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "internal - do not modify", 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:14.166 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "root", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm-package", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "root" 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:42:14.167 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "oob_default_addr": { 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "oob_default_addr", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "169.254.1.1", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.168 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "prometheus_alerts_path": { 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.169 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_insecure": { 2024-09-19T09:42:14.170 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_insecure", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.171 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_password": { 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_password", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.172 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_url": { 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_url", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.173 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_username": { 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_username", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.174 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "secure_monitoring_stack": { 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:42:14.175 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "service_discovery_port": { 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "service_discovery_port", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8765", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:42:14.177 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "ssh_config_file": { 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssh_config_file", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.178 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "use_agent": { 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_agent", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.179 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.180 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "use_repo_digest": { 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_repo_digest", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.181 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_failed_host_check": { 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:42:14.182 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.183 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_daemons": { 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.184 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_hosts": { 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.185 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crash", 2024-09-19T09:42:14.186 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.188 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.189 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.190 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.191 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "retain_interval": { 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retain_interval", 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "31536000", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_recent_interval": { 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_recent_interval", 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.196 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1209600", 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.197 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dashboard", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.198 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.199 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.200 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.201 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.202 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.203 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:42:14.204 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.205 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.206 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.207 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:42:14.209 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.211 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.212 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.213 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:42:14.214 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.215 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.216 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.217 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_URL": { 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.218 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:42:14.219 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.221 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.222 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.223 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.224 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.226 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.227 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.228 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.229 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.230 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.231 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.233 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:42:14.234 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.235 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.236 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.237 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.238 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.239 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.241 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "45", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.242 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.243 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:42:14.244 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.245 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.246 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.247 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.248 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.249 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.250 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.251 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "cross_origin_url": { 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cross_origin_url", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.252 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "crt_file": { 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crt_file", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.253 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "debug": { 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "debug", 2024-09-19T09:42:14.254 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.255 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "jwt_token_ttl": { 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "jwt_token_ttl", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "28800", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.256 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.257 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.259 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.260 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.261 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.262 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.263 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "motd": { 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "motd", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "The message of the day", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.265 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.266 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:14.267 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8080", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.270 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl_server_port": { 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl_server_port", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8443", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.271 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "redirect", 2024-09-19T09:42:14.272 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect" 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.273 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.274 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "url_prefix": { 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url_prefix", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.275 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "devicehealth", 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.276 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_monitoring": { 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_monitoring", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "monitor device health metrics", 2024-09-19T09:42:14.277 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.278 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.279 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.280 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.281 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.282 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.283 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.284 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mark_out_threshold": { 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mark_out_threshold", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2419200", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.285 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": { 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_name", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "device_health_metrics", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.286 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "retention_period": { 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retention_period", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.287 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15552000", 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_frequency": { 2024-09-19T09:42:14.288 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_frequency", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.289 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "self_heal": { 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "self_heal", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.290 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.291 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_threshold": { 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_threshold", 2024-09-19T09:42:14.292 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7257600", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.293 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "diskprediction_local", 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.295 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.296 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.297 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.298 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.299 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "predict_interval": { 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predict_interval", 2024-09-19T09:42:14.300 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.301 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "predictor_model": { 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predictor_model", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "prophetstor", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.302 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.303 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "influx", 2024-09-19T09:42:14.304 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": false, 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "batch_size": { 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "batch_size", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5000", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.305 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "database": { 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "database", 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.306 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "ceph", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": { 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hostname", 2024-09-19T09:42:14.307 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.308 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "30", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "5", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.309 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.310 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.311 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.312 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.313 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.314 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.315 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "password": { 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "password", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.316 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.317 INFO:teuthology.orchestra.run.smithi028.stdout: "port": { 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "port", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8086", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server port", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.318 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "false", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.319 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "threads": { 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threads", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.320 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "32", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "username": { 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "username", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.321 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.322 INFO:teuthology.orchestra.run.smithi028.stdout: "verify_ssl": { 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "verify_ssl", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "true", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:14.323 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "insights", 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.324 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.325 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.328 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.329 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.330 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "iostat", 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.331 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.332 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.333 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.334 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.335 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.336 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.337 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "k8sevents", 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.338 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_event_retention_days": { 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.339 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "config_check_secs": { 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_check_secs", 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "10", 2024-09-19T09:42:14.340 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.341 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.342 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.343 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.344 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.345 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.346 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "localpool", 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "failure_domain": { 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "failure_domain", 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "host", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.349 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.350 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.351 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.352 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.353 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.354 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.355 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.355 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.355 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.355 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.355 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.356 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": { 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_size", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:42:14.357 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "num_rep": { 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "num_rep", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.358 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": { 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_num", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.359 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "128", 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "prefix": { 2024-09-19T09:42:14.360 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prefix", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.361 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "subtree": { 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "subtree", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "rack", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:42:14.362 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mds_autoscaler", 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.364 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.365 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.366 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.367 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.368 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.369 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirroring", 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.370 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.371 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.372 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.373 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.374 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.376 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "nfs", 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.377 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.378 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.379 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.380 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.381 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.382 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.383 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.384 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "fail_fs": { 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "fail_fs", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.385 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.386 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.387 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.388 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.389 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.390 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.391 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator": { 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.392 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "rook", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "test_orchestrator" 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Orchestrator backend", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.393 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_perf_query", 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.394 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.395 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.396 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.398 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.400 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.401 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.402 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_support", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.403 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.404 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.405 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.406 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.407 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.408 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_autoscaler", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.410 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.411 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.412 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.413 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.414 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.415 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.416 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.417 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "threshold": { 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threshold", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1.0", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "scaling threshold", 2024-09-19T09:42:14.418 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "progress", 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.419 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_pg_recovery_event": { 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.420 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.421 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.422 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.423 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.424 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.425 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.426 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.427 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "max_completed_events": { 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_completed_events", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "50", 2024-09-19T09:42:14.428 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.429 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.430 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus", 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "cache": { 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cache", 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.431 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "exclude_perf_counters": { 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "exclude_perf_counters", 2024-09-19T09:42:14.432 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.433 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.434 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.435 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.436 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.437 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.438 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.439 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools": { 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools", 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.440 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.441 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "300", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.442 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_interval": { 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_interval", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15.0", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.443 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.444 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "9283", 2024-09-19T09:42:14.445 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "stale_cache_strategy": { 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stale_cache_strategy", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.446 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "log", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.447 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "default", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "default", 2024-09-19T09:42:14.448 INFO:teuthology.orchestra.run.smithi028.stdout: "error" 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.449 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.450 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_support", 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.451 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.452 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.453 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.454 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.455 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.456 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_snap_create": { 2024-09-19T09:42:14.457 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.458 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.459 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "trash_purge_schedule": { 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "trash_purge_schedule", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.460 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "restful", 2024-09-19T09:42:14.461 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_auth": { 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_auth", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.462 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.463 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.465 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.466 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.467 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.468 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.469 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.470 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.471 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:14.472 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.473 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rgw", 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.474 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.475 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.476 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.477 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.478 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.479 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.480 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:42:14.481 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rook", 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.482 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.483 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.484 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.485 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.486 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.487 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.488 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "storage_class": { 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "storage_class", 2024-09-19T09:42:14.489 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "local", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.490 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "selftest", 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.491 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.492 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.493 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.494 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.495 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.496 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "roption1": { 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption1", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.497 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "roption2": { 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption2", 2024-09-19T09:42:14.498 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "xyz", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption1": { 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption1", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.500 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption2": { 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption2", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.501 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption3": { 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption3", 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.502 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption4": { 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption4", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.503 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.504 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption5": { 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption5", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.505 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption6": { 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption6", 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.506 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption7": { 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption7", 2024-09-19T09:42:14.507 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "42", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "testkey": { 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testkey", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.509 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "testlkey": { 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testlkey", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.510 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "testnewline": { 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testnewline", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.511 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snap_schedule", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_m_granularity": { 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_m_granularity", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.513 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "dump_on_update": { 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dump_on_update", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.514 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.515 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.516 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.517 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.519 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.520 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.521 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stats", 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.522 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.523 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.524 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.526 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.527 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.528 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "status", 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.529 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.530 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.531 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.532 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.533 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.534 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.535 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telegraf", 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "address": { 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "address", 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.536 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.537 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.538 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.539 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.541 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.542 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.543 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.544 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telemetry", 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.545 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_basic": { 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_basic", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.546 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_crash": { 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_crash", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.547 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_device": { 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_device", 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.548 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_ident": { 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_ident", 2024-09-19T09:42:14.549 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.550 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_perf": { 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_perf", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.551 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "contact": { 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "contact", 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.552 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "description": { 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "description", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.553 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.554 INFO:teuthology.orchestra.run.smithi028.stdout: "device_url": { 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_url", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.555 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.557 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "24", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "8", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "last_opt_revision": { 2024-09-19T09:42:14.558 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "last_opt_revision", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.559 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard": { 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.560 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard_description": { 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard_description", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.561 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.562 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.563 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.564 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.565 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.566 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.567 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "organization": { 2024-09-19T09:42:14.568 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "organization", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.569 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "proxy": { 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "proxy", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.570 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "url": { 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:42:14.571 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "test_orchestrator", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.574 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.575 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.576 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.577 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.578 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "volumes", 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.579 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.580 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.582 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.583 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.584 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.585 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_clones": { 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_clones", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "periodic_async_work": { 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "periodic_async_work", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Periodically check for async work", 2024-09-19T09:42:14.588 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_delay": { 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.589 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.590 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "discovery_interval": { 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "discovery_interval", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "100", 2024-09-19T09:42:14.592 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "identifier": { 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "identifier", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.593 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.594 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.595 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.596 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.598 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.599 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.601 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_host": { 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_host", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.605 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_port": { 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_port", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10051", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.607 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_sender": { 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_sender", 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:42:14.608 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:14.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "gid": 24110, 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "y", 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "available_modules": [ 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "alerts", 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.610 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.611 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.612 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.613 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.614 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.615 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.616 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.617 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_destination": { 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_destination", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.618 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.619 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_from_name": { 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_from_name", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "Ceph", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email From: name", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.620 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_host": { 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_host", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.621 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP server", 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_password": { 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_password", 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.622 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Password to authenticate with", 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_port": { 2024-09-19T09:42:14.623 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_port", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "465", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP port", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.624 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_sender": { 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_sender", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.625 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_ssl": { 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_ssl", 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.626 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_user": { 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_user", 2024-09-19T09:42:14.627 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "User to authenticate as", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.628 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "balancer", 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "active": { 2024-09-19T09:42:14.629 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "active", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_time": { 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_time", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0000", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.631 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_weekday": { 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_weekday", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:14.632 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_max_iterations": { 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.633 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "25", 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "250", 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_metrics": { 2024-09-19T09:42:14.634 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_metrics", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.635 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_step": { 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_step", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.5", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0.001", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "0.999", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:42:14.636 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "end_time": { 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_time", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2359", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.637 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "end_weekday": { 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_weekday", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.638 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.639 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.640 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.641 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.642 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.643 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.644 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "min_score": { 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_score", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:42:14.646 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "upmap", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.647 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-compat", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "none", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "read", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap-read" 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Balancer mode", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.648 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_ids": { 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_ids", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.649 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.650 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_deviation": { 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_deviation", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.651 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_optimizations": { 2024-09-19T09:42:14.652 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.653 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm", 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_down_multiplier": { 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_down_multiplier", 2024-09-19T09:42:14.654 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.655 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_refresh_rate": { 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_refresh_rate", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "20", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.656 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_starting_port": { 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_starting_port", 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4721", 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.657 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_ptrace": { 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_ptrace", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.658 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.659 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_interval": { 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_interval", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.660 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.7", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.661 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm_log_destination": { 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm_log_destination", 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.662 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "file", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "file,syslog", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "syslog" 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.663 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "cgroups_split": { 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cgroups_split", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.664 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "config_checks_enabled": { 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_checks_enabled", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.665 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "config_dashboard": { 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_dashboard", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.666 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.667 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_alertmanager": { 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_alertmanager", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.668 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_base": { 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_base", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.669 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_elasticsearch": { 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.670 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "elasticsearch container image", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_grafana": { 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_grafana", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.671 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.672 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_haproxy": { 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_haproxy", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "HAproxy container image", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.673 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_agent": { 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.674 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_collector": { 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:42:14.675 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_query": { 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.676 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger query container image", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.677 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_keepalived": { 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_keepalived", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Keepalived container image", 2024-09-19T09:42:14.678 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_loki": { 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_loki", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.679 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Loki container image", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_node_exporter": { 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_node_exporter", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.680 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_nvmeof": { 2024-09-19T09:42:14.681 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_nvmeof", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Nvme-of container image", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.682 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_prometheus": { 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_prometheus", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:14.683 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_promtail": { 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_promtail", 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:42:14.684 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Promtail container image", 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_samba": { 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_samba", 2024-09-19T09:42:14.685 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.686 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_snmp_gateway": { 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.687 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "container_init": { 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_init", 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.688 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "daemon_cache_timeout": { 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.689 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.690 INFO:teuthology.orchestra.run.smithi028.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "900", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.691 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "default_registry": { 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_registry", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.692 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "device_cache_timeout": { 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_cache_timeout", 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.693 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1800", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "device_enhanced_scan": { 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_enhanced_scan", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.694 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.695 INFO:teuthology.orchestra.run.smithi028.stdout: "facts_cache_timeout": { 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "facts_cache_timeout", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.696 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "grafana_dashboards_path": { 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.697 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "host_check_interval": { 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "host_check_interval", 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.698 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "hw_monitoring": { 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hw_monitoring", 2024-09-19T09:42:14.699 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "inventory_list_all": { 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "inventory_list_all", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.701 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.702 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.703 INFO:teuthology.orchestra.run.smithi028.stdout: "log_refresh_metadata": { 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_refresh_metadata", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:42:14.704 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.705 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.706 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.707 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.708 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:42:14.709 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.710 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "*", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.711 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "max_count_per_host": { 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_count_per_host", 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.712 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd_draining_count": { 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_osd_draining_count", 2024-09-19T09:42:14.713 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.714 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "migration_current": { 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "migration_current", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.715 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "internal - do not modify", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "root", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.716 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm-package", 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "root" 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "oob_default_addr": { 2024-09-19T09:42:14.717 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "oob_default_addr", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "169.254.1.1", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.718 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "prometheus_alerts_path": { 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.719 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_insecure": { 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_insecure", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.720 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_password": { 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_password", 2024-09-19T09:42:14.721 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.722 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_url": { 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_url", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:42:14.723 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_username": { 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_username", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.724 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "secure_monitoring_stack": { 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.725 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "service_discovery_port": { 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "service_discovery_port", 2024-09-19T09:42:14.726 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8765", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.727 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "ssh_config_file": { 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssh_config_file", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:42:14.728 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "use_agent": { 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_agent", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.729 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "use_repo_digest": { 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_repo_digest", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.730 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_failed_host_check": { 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:42:14.731 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.732 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_daemons": { 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:42:14.733 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_hosts": { 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.734 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crash", 2024-09-19T09:42:14.735 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.736 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.737 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.738 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.739 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.740 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.741 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "retain_interval": { 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retain_interval", 2024-09-19T09:42:14.742 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "31536000", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.743 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_recent_interval": { 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_recent_interval", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1209600", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.744 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dashboard", 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.745 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.746 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.747 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.749 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.750 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.751 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.752 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.753 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.754 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.755 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.756 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.757 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:42:14.758 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.759 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.760 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.764 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.765 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_URL": { 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.766 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:42:14.767 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.768 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.769 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.770 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.771 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.773 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.774 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.775 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.776 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.777 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.778 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.779 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:42:14.780 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.781 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.782 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.783 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:42:14.784 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.785 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.786 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.787 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.788 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.789 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.790 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "45", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.791 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.792 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.793 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:42:14.794 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.795 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.796 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:14.797 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.798 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:42:14.799 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.800 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "cross_origin_url": { 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cross_origin_url", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.801 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "crt_file": { 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crt_file", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.802 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "debug": { 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "debug", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.803 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "jwt_token_ttl": { 2024-09-19T09:42:14.804 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "jwt_token_ttl", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "28800", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.805 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.808 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.809 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.810 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.811 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.812 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.813 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.814 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.815 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "motd": { 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "motd", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.816 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "The message of the day", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.817 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:14.818 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.819 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8080", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl_server_port": { 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl_server_port", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8443", 2024-09-19T09:42:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "redirect", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect" 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "url_prefix": { 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url_prefix", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.827 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "devicehealth", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_monitoring": { 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_monitoring", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.828 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "monitor device health metrics", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.829 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.835 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "mark_out_threshold": { 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mark_out_threshold", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2419200", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:42:14.836 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": { 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_name", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "device_health_metrics", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.837 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "retention_period": { 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retention_period", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.838 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15552000", 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_frequency": { 2024-09-19T09:42:14.839 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_frequency", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.840 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "self_heal": { 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "self_heal", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.841 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.842 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_threshold": { 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_threshold", 2024-09-19T09:42:14.843 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7257600", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.844 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "diskprediction_local", 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.845 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.846 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.847 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.848 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.849 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.850 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.851 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "predict_interval": { 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predict_interval", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.852 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "predictor_model": { 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predictor_model", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.853 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "prophetstor", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.854 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.855 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "influx", 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": false, 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "batch_size": { 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "batch_size", 2024-09-19T09:42:14.856 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5000", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.857 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "database": { 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "database", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "ceph", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.858 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": { 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hostname", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.859 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:14.860 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "30", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "5", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.861 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.862 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.863 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.864 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.867 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "password": { 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "password", 2024-09-19T09:42:14.868 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.869 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "port": { 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "port", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8086", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.870 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server port", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "false", 2024-09-19T09:42:14.871 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "threads": { 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threads", 2024-09-19T09:42:14.872 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "32", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.873 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "username": { 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "username", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:42:14.874 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "verify_ssl": { 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "verify_ssl", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "true", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.875 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "insights", 2024-09-19T09:42:14.876 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.877 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.878 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.879 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.880 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.881 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.882 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.883 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "iostat", 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.884 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.885 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.886 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.887 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.888 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.889 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.890 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "k8sevents", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_event_retention_days": { 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7", 2024-09-19T09:42:14.891 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "config_check_secs": { 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_check_secs", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.892 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "10", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.893 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.894 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.895 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.896 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "localpool", 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "failure_domain": { 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "failure_domain", 2024-09-19T09:42:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "host", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": { 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_size", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "num_rep": { 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "num_rep", 2024-09-19T09:42:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:42:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": { 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_num", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "128", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "prefix": { 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prefix", 2024-09-19T09:42:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "subtree": { 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "subtree", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "rack", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:42:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mds_autoscaler", 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirroring", 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "nfs", 2024-09-19T09:42:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "fail_fs": { 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "fail_fs", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator": { 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm", 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "rook", 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "test_orchestrator" 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Orchestrator backend", 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_perf_query", 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_support", 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_autoscaler", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "threshold": { 2024-09-19T09:42:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threshold", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1.0", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "scaling threshold", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:42:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "progress", 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_pg_recovery_event": { 2024-09-19T09:42:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "max_completed_events": { 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_completed_events", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "50", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "cache": { 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cache", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "exclude_perf_counters": { 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "exclude_perf_counters", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.992 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools": { 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.993 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.994 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "300", 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_interval": { 2024-09-19T09:42:14.995 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_interval", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15.0", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.996 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:42:14.997 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "9283", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:14.998 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "stale_cache_strategy": { 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stale_cache_strategy", 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:14.999 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "log", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:15.000 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "default", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "default", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "error" 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.001 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:15.002 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_support", 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.003 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.004 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.005 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.007 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.008 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.009 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.010 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_snap_create": { 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.012 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.014 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "trash_purge_schedule": { 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "trash_purge_schedule", 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.015 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "restful", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_auth": { 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_auth", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.017 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:15.018 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.019 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.020 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.021 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.022 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.023 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.024 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.025 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:15.026 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.028 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rgw", 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.029 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.030 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.032 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.033 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.034 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.035 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.036 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rook", 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.038 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.039 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.040 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.041 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.042 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.043 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.044 INFO:teuthology.orchestra.run.smithi028.stdout: "storage_class": { 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "storage_class", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "local", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.045 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "selftest", 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.046 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.047 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.048 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.050 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.052 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "roption1": { 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption1", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.053 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "roption2": { 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption2", 2024-09-19T09:42:15.054 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "xyz", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.055 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption1": { 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption1", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.056 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption2": { 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption2", 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.057 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption3": { 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption3", 2024-09-19T09:42:15.058 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.059 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption4": { 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption4", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.060 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption5": { 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption5", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.061 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption6": { 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption6", 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.062 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption7": { 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption7", 2024-09-19T09:42:15.063 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "42", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.064 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "testkey": { 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testkey", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.065 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "testlkey": { 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testlkey", 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.066 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.067 INFO:teuthology.orchestra.run.smithi028.stdout: "testnewline": { 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testnewline", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.068 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snap_schedule", 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.069 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_m_granularity": { 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_m_granularity", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.070 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "dump_on_update": { 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dump_on_update", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.071 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.072 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.073 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.074 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.075 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.076 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.077 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stats", 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.079 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.080 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.081 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.082 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.083 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.084 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.085 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "status", 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.086 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.087 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.088 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.089 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.090 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.091 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.092 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telegraf", 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "address": { 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "address", 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.093 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.094 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.095 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.096 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.098 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.099 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.100 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.101 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telemetry", 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.102 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_basic": { 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_basic", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.103 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_crash": { 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_crash", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.104 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_device": { 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_device", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.105 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_ident": { 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_ident", 2024-09-19T09:42:15.106 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.107 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_perf": { 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_perf", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.108 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "contact": { 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "contact", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.109 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "description": { 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "description", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.110 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.111 INFO:teuthology.orchestra.run.smithi028.stdout: "device_url": { 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_url", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.112 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.113 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "24", 2024-09-19T09:42:15.114 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "8", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "last_opt_revision": { 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "last_opt_revision", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.115 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.116 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard": { 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.117 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard_description": { 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard_description", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.118 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.119 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.120 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.121 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.122 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.123 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.124 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "organization": { 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "organization", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.125 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "proxy": { 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "proxy", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.126 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "url": { 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.127 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.128 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "test_orchestrator", 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.129 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.130 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.131 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.132 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.133 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.134 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.135 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "volumes", 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.136 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.137 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.138 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.139 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.140 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.141 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_clones": { 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_clones", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.142 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4", 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: "periodic_async_work": { 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "periodic_async_work", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Periodically check for async work", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.144 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_delay": { 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.145 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.146 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.147 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "discovery_interval": { 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "discovery_interval", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "100", 2024-09-19T09:42:15.148 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "identifier": { 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "identifier", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.149 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.150 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.151 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.152 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.153 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.154 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.155 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.156 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.157 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_host": { 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_host", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.158 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_port": { 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_port", 2024-09-19T09:42:15.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10051", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.160 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_sender": { 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_sender", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.161 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: "modules": [ 2024-09-19T09:42:15.162 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "dashboard", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "iostat", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "nfs", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "restful" 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "available_modules": [ 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "alerts", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.163 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.164 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.165 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.166 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.167 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.168 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.169 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.170 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_destination": { 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_destination", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.171 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_from_name": { 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_from_name", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.172 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "Ceph", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Email From: name", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.173 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_host": { 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_host", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP server", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.174 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_password": { 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_password", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.175 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Password to authenticate with", 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_port": { 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_port", 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.176 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "465", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP port", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_sender": { 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_sender", 2024-09-19T09:42:15.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.178 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_ssl": { 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_ssl", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.179 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "smtp_user": { 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "smtp_user", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.180 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "User to authenticate as", 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "balancer", 2024-09-19T09:42:15.181 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "active": { 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "active", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.182 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_time": { 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_time", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0000", 2024-09-19T09:42:15.183 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "begin_weekday": { 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "begin_weekday", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.184 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_max_iterations": { 2024-09-19T09:42:15.185 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "25", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "250", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.186 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_metrics": { 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_metrics", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.187 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_compat_step": { 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crush_compat_step", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.5", 2024-09-19T09:42:15.188 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0.001", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "0.999", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "end_time": { 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_time", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.189 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2359", 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "end_weekday": { 2024-09-19T09:42:15.190 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "end_weekday", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "0", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "6", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.192 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.194 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.195 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.196 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.197 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "min_score": { 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_score", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.198 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.199 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "upmap", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-compat", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "none", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "read", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap", 2024-09-19T09:42:15.200 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap-read" 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Balancer mode", 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_ids": { 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_ids", 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.201 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.203 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_deviation": { 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_deviation", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:42:15.204 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "upmap_max_optimizations": { 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.205 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm", 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_down_multiplier": { 2024-09-19T09:42:15.206 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_down_multiplier", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.207 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_refresh_rate": { 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_refresh_rate", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "20", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.208 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "agent_starting_port": { 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "agent_starting_port", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4721", 2024-09-19T09:42:15.209 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_ptrace": { 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_ptrace", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.210 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.211 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_interval": { 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_interval", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.212 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0.7", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.213 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:42:15.214 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm_log_destination": { 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cephadm_log_destination", 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.216 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "file", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "file,syslog", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "syslog" 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.217 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "cgroups_split": { 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cgroups_split", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.218 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "config_checks_enabled": { 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_checks_enabled", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.219 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "config_dashboard": { 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_dashboard", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.220 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.221 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_alertmanager": { 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_alertmanager", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.222 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_base": { 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_base", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.223 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_elasticsearch": { 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:42:15.224 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "elasticsearch container image", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_grafana": { 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_grafana", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.225 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_haproxy": { 2024-09-19T09:42:15.226 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_haproxy", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "HAproxy container image", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.227 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_agent": { 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.228 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_collector": { 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:42:15.229 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_jaeger_query": { 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.230 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Jaeger query container image", 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_keepalived": { 2024-09-19T09:42:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_keepalived", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Keepalived container image", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.232 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_loki": { 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_loki", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Loki container image", 2024-09-19T09:42:15.233 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_node_exporter": { 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_node_exporter", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.234 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_nvmeof": { 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_nvmeof", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.235 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Nvme-of container image", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_prometheus": { 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_prometheus", 2024-09-19T09:42:15.236 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Prometheus container image", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_promtail": { 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_promtail", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Promtail container image", 2024-09-19T09:42:15.238 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_samba": { 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_samba", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.239 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image_snmp_gateway": { 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.240 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "container_init": { 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "container_init", 2024-09-19T09:42:15.241 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "daemon_cache_timeout": { 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:42:15.243 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "900", 2024-09-19T09:42:15.244 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "default_registry": { 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "default_registry", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.245 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "docker.io", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.246 INFO:teuthology.orchestra.run.smithi028.stdout: "device_cache_timeout": { 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_cache_timeout", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1800", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.247 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "device_enhanced_scan": { 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_enhanced_scan", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.248 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "facts_cache_timeout": { 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "facts_cache_timeout", 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.249 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "grafana_dashboards_path": { 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:42:15.250 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.251 INFO:teuthology.orchestra.run.smithi028.stdout: "host_check_interval": { 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "host_check_interval", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.252 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "hw_monitoring": { 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hw_monitoring", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.253 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "inventory_list_all": { 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "inventory_list_all", 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.254 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.255 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.256 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "log_refresh_metadata": { 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_refresh_metadata", 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.257 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.258 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.259 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.260 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.261 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:42:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "*", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.265 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "max_count_per_host": { 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_count_per_host", 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.266 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd_draining_count": { 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_osd_draining_count", 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.267 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "migration_current": { 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "migration_current", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.269 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "internal - do not modify", 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": { 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mode", 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.270 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "root", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm-package", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "root" 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.271 INFO:teuthology.orchestra.run.smithi028.stdout: "oob_default_addr": { 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "oob_default_addr", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "169.254.1.1", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.272 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "prometheus_alerts_path": { 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.273 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_insecure": { 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_insecure", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.274 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_password": { 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_password", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.275 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.276 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_url": { 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_url", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.277 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "registry_username": { 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "registry_username", 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.278 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "secure_monitoring_stack": { 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.279 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.280 INFO:teuthology.orchestra.run.smithi028.stdout: "service_discovery_port": { 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "service_discovery_port", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8765", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.281 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "ssh_config_file": { 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssh_config_file", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.282 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "use_agent": { 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_agent", 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.283 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "use_repo_digest": { 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "use_repo_digest", 2024-09-19T09:42:15.284 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.285 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_failed_host_check": { 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:42:15.286 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_daemons": { 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.287 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_on_stray_hosts": { 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.288 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.289 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crash", 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.290 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.291 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.292 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.293 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.294 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.295 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "retain_interval": { 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retain_interval", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.296 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "31536000", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.297 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_recent_interval": { 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_recent_interval", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1209600", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.298 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dashboard", 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:42:15.299 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.300 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.301 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.302 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.303 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:42:15.304 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.306 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.307 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.308 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.309 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.310 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.311 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.312 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:42:15.313 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.314 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.315 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.316 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:42:15.317 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.318 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.319 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_URL": { 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.320 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.321 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:42:15.322 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.327 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.328 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.329 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.330 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:42:15.331 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.332 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.334 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.335 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.336 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.338 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.339 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:42:15.340 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.341 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.342 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.343 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "45", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.344 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.345 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:42:15.346 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "admin", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.347 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.348 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.349 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.350 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.351 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:42:15.352 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.353 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "cross_origin_url": { 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cross_origin_url", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.354 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "crt_file": { 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "crt_file", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.355 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "debug": { 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "debug", 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.356 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "jwt_token_ttl": { 2024-09-19T09:42:15.357 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "jwt_token_ttl", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "28800", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.358 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.359 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.360 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.362 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.363 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.364 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.365 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "motd": { 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "motd", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.366 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "The message of the day", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.367 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.368 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.369 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.370 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8080", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.371 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.372 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl_server_port": { 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl_server_port", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8443", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "redirect", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.374 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "redirect" 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.375 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.376 INFO:teuthology.orchestra.run.smithi028.stdout: "url_prefix": { 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url_prefix", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.377 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "devicehealth", 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_monitoring": { 2024-09-19T09:42:15.378 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_monitoring", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "monitor device health metrics", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.379 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.380 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.381 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.382 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.383 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.384 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.385 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "mark_out_threshold": { 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mark_out_threshold", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "2419200", 2024-09-19T09:42:15.386 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": { 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pool_name", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.387 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "device_health_metrics", 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "retention_period": { 2024-09-19T09:42:15.388 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "retention_period", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15552000", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.389 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_frequency": { 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_frequency", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.390 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "self_heal": { 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "self_heal", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.391 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:15.392 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "warn_threshold": { 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "warn_threshold", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7257600", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.394 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "diskprediction_local", 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.395 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.396 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.397 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.399 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.400 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.401 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "predict_interval": { 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predict_interval", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.402 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "86400", 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "predictor_model": { 2024-09-19T09:42:15.403 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "predictor_model", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "prophetstor", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.404 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "600", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.405 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "influx", 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": false, 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:42:15.406 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "batch_size": { 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "batch_size", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5000", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:42:15.407 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "database": { 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "database", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "ceph", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.408 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": { 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "hostname", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.409 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.410 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "30", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "5", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.411 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.412 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.413 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.414 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.415 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.416 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.417 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "password": { 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "password", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.418 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "port": { 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "port", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "8086", 2024-09-19T09:42:15.419 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "InfluxDB server port", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "ssl": { 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ssl", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.420 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "false", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.421 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "threads": { 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threads", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "32", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.422 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "username": { 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "username", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "verify_ssl": { 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "verify_ssl", 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.424 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "true", 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.425 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "insights", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.426 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.427 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.428 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.429 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.430 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.431 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.432 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "iostat", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.433 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.434 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.435 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.436 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.437 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.438 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.439 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "k8sevents", 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_event_retention_days": { 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.440 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "7", 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "config_check_secs": { 2024-09-19T09:42:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "config_check_secs", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "10", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.442 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.443 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.444 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.445 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.446 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.447 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.448 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "localpool", 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.449 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "failure_domain": { 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "failure_domain", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "host", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.450 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.451 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.452 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.453 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.454 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.455 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.456 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": { 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "min_size", 2024-09-19T09:42:15.457 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.458 INFO:teuthology.orchestra.run.smithi028.stdout: "num_rep": { 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "num_rep", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.459 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": { 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_num", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "128", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.460 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "prefix": { 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prefix", 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.461 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "subtree": { 2024-09-19T09:42:15.462 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "subtree", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "rack", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mds_autoscaler", 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.464 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.465 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.466 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.467 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.468 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.469 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.470 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.471 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirroring", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.472 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.473 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.474 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "nfs", 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.479 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.480 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.481 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.483 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.484 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "fail_fs": { 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "fail_fs", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.486 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.487 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.488 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.489 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.490 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.491 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.492 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.493 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator": { 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "orchestrator", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.494 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm", 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "rook", 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "test_orchestrator" 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Orchestrator backend", 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_perf_query", 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.496 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.497 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.498 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.499 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.500 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.501 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.502 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "osd_support", 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.503 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.504 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.505 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.506 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.507 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.508 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.509 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "pg_autoscaler", 2024-09-19T09:42:15.510 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.511 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.512 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.513 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.514 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.515 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.516 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:15.517 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.518 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "threshold": { 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "threshold", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "3.0", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1.0", 2024-09-19T09:42:15.519 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "scaling threshold", 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.520 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "progress", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_pg_recovery_event": { 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.521 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.522 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.523 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.524 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.525 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.527 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.529 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "max_completed_events": { 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_completed_events", 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "50", 2024-09-19T09:42:15.530 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "sleep_interval": { 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "sleep_interval", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.531 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "prometheus", 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "cache": { 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "cache", 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.533 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "exclude_perf_counters": { 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "exclude_perf_counters", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.534 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.536 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.537 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.538 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.539 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.540 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.541 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools": { 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.542 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "300", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.543 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "scrape_interval": { 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "scrape_interval", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.544 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15.0", 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:15.545 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "::", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.546 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "9283", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.547 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "stale_cache_strategy": { 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stale_cache_strategy", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "log", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.548 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_behaviour": { 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_behaviour", 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.549 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "default", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "default", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "error" 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.550 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "standby_error_status_code": { 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "standby_error_status_code", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "500", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "400", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "599", 2024-09-19T09:42:15.551 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_support", 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.552 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.553 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.554 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.556 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.557 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.558 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_snap_create": { 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.559 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.560 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "trash_purge_schedule": { 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "trash_purge_schedule", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.561 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.562 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "restful", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_auth": { 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enable_auth", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.563 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "key_file": { 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "key_file", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.564 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.565 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.566 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.567 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.569 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "server_addr": { 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_addr", 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.572 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "server_port": { 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "server_port", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.573 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rgw", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.575 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.576 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.577 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.578 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.579 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.580 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:42:15.581 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "5", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.582 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rook", 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.583 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.585 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.586 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.587 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.588 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.589 INFO:teuthology.orchestra.run.smithi028.stdout: "storage_class": { 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "storage_class", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "local", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.590 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "selftest", 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.591 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.592 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.593 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.594 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.595 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.597 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "roption1": { 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption1", 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.598 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "roption2": { 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "roption2", 2024-09-19T09:42:15.599 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "xyz", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption1": { 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption1", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.601 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption2": { 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption2", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.602 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption3": { 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption3", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "float", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.603 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption4": { 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption4", 2024-09-19T09:42:15.604 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.605 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption5": { 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption5", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption6": { 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption6", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.607 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "rwoption7": { 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rwoption7", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "1", 2024-09-19T09:42:15.608 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "42", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "testkey": { 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testkey", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.609 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "testlkey": { 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testlkey", 2024-09-19T09:42:15.610 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.611 INFO:teuthology.orchestra.run.smithi028.stdout: "testnewline": { 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "testnewline", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.612 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snap_schedule", 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.613 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_m_granularity": { 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "allow_m_granularity", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:42:15.614 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "dump_on_update": { 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "dump_on_update", 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.615 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.616 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.617 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.620 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.621 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.621 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.621 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.624 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.625 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "stats", 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.626 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.627 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.628 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.629 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.630 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.631 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.632 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "status", 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.633 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.634 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.635 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.636 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.637 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.638 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.639 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telegraf", 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.640 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "address": { 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "address", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.641 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.642 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "15", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.643 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.644 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.645 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.646 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.648 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.649 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "telemetry", 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_basic": { 2024-09-19T09:42:15.650 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_basic", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.651 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_crash": { 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_crash", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.652 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_device": { 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_device", 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.653 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_ident": { 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_ident", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.654 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.655 INFO:teuthology.orchestra.run.smithi028.stdout: "channel_perf": { 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "channel_perf", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.656 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "contact": { 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "contact", 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.657 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "description": { 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "description", 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.658 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "device_url": { 2024-09-19T09:42:15.659 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "device_url", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.660 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "enabled": { 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "enabled", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.661 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "24", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "8", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.662 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "last_opt_revision": { 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "last_opt_revision", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "1", 2024-09-19T09:42:15.663 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard": { 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard", 2024-09-19T09:42:15.664 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.665 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "leaderboard_description": { 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "leaderboard_description", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.666 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.667 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.669 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.670 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.671 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.672 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "organization": { 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "organization", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.673 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.674 INFO:teuthology.orchestra.run.smithi028.stdout: "proxy": { 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "proxy", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.675 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "url": { 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "url", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.676 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "test_orchestrator", 2024-09-19T09:42:15.677 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.678 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.679 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.680 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.681 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.682 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.683 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.684 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "volumes", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.685 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.686 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.687 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.688 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.689 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.690 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "max_concurrent_clones": { 2024-09-19T09:42:15.691 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "max_concurrent_clones", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "4", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.692 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "periodic_async_work": { 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "periodic_async_work", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.693 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Periodically check for async work", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_delay": { 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "0", 2024-09-19T09:42:15.694 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.695 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "True", 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.696 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "can_run": true, 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "error_string": "", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "module_options": { 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "discovery_interval": { 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "discovery_interval", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "uint", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "100", 2024-09-19T09:42:15.697 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "identifier": { 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "identifier", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.698 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "interval": { 2024-09-19T09:42:15.699 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "interval", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "secs", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "60", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.700 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "log_level": { 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_level", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.701 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.702 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster": { 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.703 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_cluster_level": { 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_cluster_level", 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "info", 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.704 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [ 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "critical", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "debug", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "error", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "info", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "warning" 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.705 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": { 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "log_to_file", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "bool", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "False", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.706 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_host": { 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_host", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.707 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_port": { 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_port", 2024-09-19T09:42:15.708 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "int", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "10051", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.709 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "zabbix_sender": { 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "zabbix_sender", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "str", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "level": "advanced", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 0, 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "min": "", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "max": "", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "enum_allowed": [], 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "desc": "", 2024-09-19T09:42:15.710 INFO:teuthology.orchestra.run.smithi028.stdout: "long_desc": "", 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: "tags": [], 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: "see_also": [] 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: "services": { 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: "dashboard": "https://172.21.15.28:8443/" 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.711 INFO:teuthology.orchestra.run.smithi028.stdout: "always_on_modules": { 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "octopus": [ 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "balancer", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "devicehealth", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscaler", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "progress", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_support", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "status", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "telemetry", 2024-09-19T09:42:15.712 INFO:teuthology.orchestra.run.smithi028.stdout: "volumes" 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "pacific": [ 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "balancer", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "devicehealth", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscaler", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "progress", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_support", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "status", 2024-09-19T09:42:15.713 INFO:teuthology.orchestra.run.smithi028.stdout: "telemetry", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "volumes" 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "quincy": [ 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "balancer", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "devicehealth", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscaler", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "progress", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_support", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "status", 2024-09-19T09:42:15.714 INFO:teuthology.orchestra.run.smithi028.stdout: "telemetry", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "volumes" 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "reef": [ 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "balancer", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "devicehealth", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscaler", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "progress", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_support", 2024-09-19T09:42:15.715 INFO:teuthology.orchestra.run.smithi028.stdout: "status", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "telemetry", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "volumes" 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "squid": [ 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "balancer", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "crash", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "devicehealth", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "orchestrator", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscaler", 2024-09-19T09:42:15.716 INFO:teuthology.orchestra.run.smithi028.stdout: "progress", 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_support", 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "status", 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "telemetry", 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "volumes" 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "last_failure_osd_epoch": 3, 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "active_clients": [ 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "devicehealth", 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-09-19T09:42:15.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:0", 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4252722596 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "libcephsqlite", 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-09-19T09:42:15.718 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:0", 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 193990680 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "rbd_support", 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:0", 2024-09-19T09:42:15.719 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 21720899 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "volumes", 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:0", 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 390332578 2024-09-19T09:42:15.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.721 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.721 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:15.721 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-19T09:42:15.721 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/930948801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3507689110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/336304980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:13 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4058207861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:14 smithi038 ceph-mon[35271]: pgmap v628: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 860 KiB/s wr, 337 op/s 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:14 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/630988803' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/930948801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:15.723 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3507689110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.724 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/336304980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.724 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:13 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4058207861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:42:15.724 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:14 smithi078 ceph-mon[35619]: pgmap v628: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 860 KiB/s wr, 337 op/s 2024-09-19T09:42:15.724 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:14 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/630988803' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:42:15.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 msgr2=0x7f9d680a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 0x7f9d680a5710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009c20 tx=0x7f9d78005f90 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 msgr2=0x7f9d7c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009230 tx=0x7f9d6c031b60 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 shutdown_connections 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f9d680a3250 0x7f9d680a5710 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d7c1016a0 0x7f9d7c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d7c100d00 0x7f9d7c19a5e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 --2- 172.21.15.28:0/630988803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7c1003c0 0x7f9d7c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 >> 172.21.15.28:0/630988803 conn(0x7f9d7c0fc1b0 msgr2=0x7f9d7c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 shutdown_connections 2024-09-19T09:42:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:14.049+0000 7f9d83df6640 1 -- 172.21.15.28:0/630988803 wait complete. 2024-09-19T09:42:15.731 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T09:42:15.732 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T09:42:15.732 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:14 smithi028 ceph-mon[30972]: pgmap v628: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 860 KiB/s wr, 337 op/s 2024-09-19T09:42:15.732 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:14 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/630988803' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:42:16.000 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:16.464 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 1448, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 1448, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 4346195, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 459641, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 6, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 2, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.143414288, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.040585513, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.041274144, 2024-09-19T09:42:16.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.050072601, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 1447, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.074579243, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000051540 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:16.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 814, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 788, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 1052692, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 305452, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.083182062, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.027399663, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.036442754, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.021394015, 2024-09-19T09:42:16.467 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 787, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.051221774, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000065084 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 907392, 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 118400 2024-09-19T09:42:16.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 2, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 146, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 428, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 335086, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.011032531, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.008185107, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.001554522, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000039097, 2024-09-19T09:42:16.469 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 144, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.006227420, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000043245 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 334768 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.470 INFO:teuthology.orchestra.run.smithi028.stdout: "cct": { 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "total_workers": 1, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "unhealthy_workers": 0 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "finisher-MDSRank": { 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "queue_len": 0, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "complete_latency": { 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 3370, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.088258604, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000026189 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.471 INFO:teuthology.orchestra.run.smithi028.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "queue_len": 0, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "complete_latency": { 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "request": 0, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "reply": 0, 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "reply_latency": { 2024-09-19T09:42:16.472 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "slow_reply": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "forward": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_commit": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_split": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_merge": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes": 499, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_top": 0, 2024-09-19T09:42:16.473 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_bottom": 499, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_pinned": 499, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_expired": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "subtrees": 1, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_hit": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_forward": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_discover": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_lock": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "load_cent": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "q": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "exported": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "exported_inodes": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "imported": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "imported_inodes": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rfiles": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rbytes": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rsnaps": 0, 2024-09-19T09:42:16.475 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:16.476 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_caps": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_cap_release": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "process_request_cap_release": 0 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache": { 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_update": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_try_discover": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_send_discover": 0, 2024-09-19T09:42:16.477 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_created": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_enqueued": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_migrated": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_started": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_completed": 0, 2024-09-19T09:42:16.478 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_flush": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_inodestats": 0 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log": { 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "evlrg": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "evadd": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "evex": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "evtrm": 0, 2024-09-19T09:42:16.479 INFO:teuthology.orchestra.run.smithi028.stdout: "ev": 2852, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "evexg": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "evexd": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segadd": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segex": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segtrm": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "seg": 4, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segmjr": 2, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segexg": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "segexd": 0, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "expos": 4194304, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "wrpos": 8110088, 2024-09-19T09:42:16.480 INFO:teuthology.orchestra.run.smithi028.stdout: "rdpos": 8108950, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "jlat": { 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "replayed": 2852 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_mem": { 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 6, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "ino+": 517, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "ino-": 511, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "dir": 4, 2024-09-19T09:42:16.481 INFO:teuthology.orchestra.run.smithi028.stdout: "dir+": 36, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dir-": 32, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dn": 499, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dn+": 1219, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dn-": 720, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "cap": 0, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "cap+": 0, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "cap-": 0, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "rss": 52064, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "heap": 215328 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_server": { 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dispatch_client_request": 0, 2024-09-19T09:42:16.482 INFO:teuthology.orchestra.run.smithi028.stdout: "dispatch_server_request": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_request": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_session": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_peer_request": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "req_create_latency": { 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getattr_latency": { 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.483 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "req_link_latency": { 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookup_latency": { 2024-09-19T09:42:16.484 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupino_latency": { 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupname_latency": { 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.486 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lssnap_latency": { 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.487 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mkdir_latency": { 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mknod_latency": { 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.488 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mksnap_latency": { 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "req_open_latency": { 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "req_readdir_latency": { 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.489 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rename_latency": { 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:16.490 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmdir_latency": { 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setattr_latency": { 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.492 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setlayout_latency": { 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setxattr_latency": { 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "req_symlink_latency": { 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "req_unlink_latency": { 2024-09-19T09:42:16.494 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_sessions": { 2024-09-19T09:42:16.495 INFO:teuthology.orchestra.run.smithi028.stdout: "session_count": 1, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "session_add": 2, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "session_remove": 1, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "sessions_open": 1, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "sessions_stale": 0, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "total_load": 0, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "average_load": 0, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "avg_session_uptime": 1605, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "mempool": { 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:16.496 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:16.497 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:16.498 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_bytes": 189383, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_items": 50, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bytes": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_items": 0, 2024-09-19T09:42:16.499 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_items": 152, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_items": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_bytes": 370360, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_items": 2047, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:16.500 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_items": 0, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_items": 0 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter": { 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_active": 0, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_laggy": 0, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_send": 2033, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_send_bytes": 21, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_resend": 3, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_reply": 2033, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "op_latency": { 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 2033, 2024-09-19T09:42:16.501 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 1.990945242, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000979313 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op_inflight": 0, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "oplen_avg": { 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 2033, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 2034 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op": 2033, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op_r": 2033, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op_w": 0, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op_rmw": 0, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "op_pg": 0, 2024-09-19T09:42:16.502 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_stat": 1265, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_create": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_read": 766, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_write": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_writefull": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_writesame": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_append": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_zero": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_truncate": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_delete": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_mapext": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:16.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_getxattr": 1, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_setxattr": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_call": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_watch": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_notify": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_pgls": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_other": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_active": 0, 2024-09-19T09:42:16.504 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_send": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_resend": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_ping": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_active": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_send": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_resend": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_active": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_send": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_resend": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_active": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_send": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_resend": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "command_active": 0, 2024-09-19T09:42:16.505 INFO:teuthology.orchestra.run.smithi028.stdout: "command_send": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "command_resend": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "map_epoch": 85, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "map_full": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "map_inc": 5, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_sessions": 5, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_session_open": 5, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_session_close": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_laggy": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_wr": 0, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_rd": 2, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_del": 0 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "oft": { 2024-09-19T09:42:16.506 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_objs": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_updates": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_removes": 0 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "osd-slow-ops": { 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "slow_ops_count": 0 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "purge_queue": { 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:16.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executed": 0, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 2260, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 5181355, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 2260, 2024-09-19T09:42:16.508 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2260, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 5181355, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:16.509 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 2033, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 3915789, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2033, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 3915789, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.510 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 1024, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 2033, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 2033, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2033, 2024-09-19T09:42:16.511 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 2033, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 3758096384, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:16.512 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-write_buf_throttle-0x562feb3226e0": { 2024-09-19T09:42:16.513 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 3758096384, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:16.514 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:16.515 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:16.515 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.515 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:16.515 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:17.292 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T09:42:17.292 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T09:42:17.292 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T09:42:17.293 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-19T09:42:17.293 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-19T09:42:17.293 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T09:42:17.293 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T09:42:17.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:17 smithi028 ceph-mon[30972]: pgmap v629: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 742 KiB/s wr, 275 op/s 2024-09-19T09:42:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:17.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:17 smithi078 ceph-mon[35619]: pgmap v629: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 742 KiB/s wr, 275 op/s 2024-09-19T09:42:17.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:17 smithi038 ceph-mon[35271]: pgmap v629: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 742 KiB/s wr, 275 op/s 2024-09-19T09:42:18.170 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:18.170 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 4, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 4, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 3072, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 448, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 0, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.003507687, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.000051799, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.002525301, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:18.171 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 3, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.004457631, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.001485877 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.172 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 1, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 6, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 160, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 79632, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 3, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 2, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.002079103, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.000415491, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.001573864, 2024-09-19T09:42:18.173 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 4, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000164843, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000041210 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:42:18.174 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 79632 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 11, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 10, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 143984, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 4176, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 2, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 1, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.002347196, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.000314478, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.003215838, 2024-09-19T09:42:18.175 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 9, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000898662, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000099851 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 143984, 2024-09-19T09:42:18.176 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 4176 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "cct": { 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "total_workers": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "unhealthy_workers": 0 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "mempool": { 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:18.177 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:18.178 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:18.179 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_items": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_bytes": 148056, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_items": 26, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:18.180 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_items": 0, 2024-09-19T09:42:18.181 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_bytes": 928, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_items": 2, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_items": 0, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_items": 0 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 12, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 142496, 2024-09-19T09:42:18.182 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 12, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 12, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 142496, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:18.183 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:18.950 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T09:42:18.950 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T09:42:18.950 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T09:42:18.950 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T09:42:18.950 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-19T09:42:18.951 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T09:42:18.951 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T09:42:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:19.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:19 smithi078 ceph-mon[35619]: pgmap v630: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 622 KiB/s wr, 232 op/s 2024-09-19T09:42:19.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:19 smithi038 ceph-mon[35271]: pgmap v630: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 622 KiB/s wr, 232 op/s 2024-09-19T09:42:19.708 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:19 smithi028 ceph-mon[30972]: pgmap v630: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 622 KiB/s wr, 232 op/s 2024-09-19T09:42:19.813 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 1104, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 1078, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 2779101, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 398248, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 1, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.130577091, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.048829698, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.043369784, 2024-09-19T09:42:19.814 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.026099567, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 1077, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.062598558, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000058123 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 907936, 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 119680 2024-09-19T09:42:19.815 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 1223, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 1368, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 12894243, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 723597, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.110908381, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.031204779, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.036396917, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.036260673, 2024-09-19T09:42:19.816 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 1366, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.070884087, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000051891 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.817 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 334944 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 8, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 8, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 4701, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 1732, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:19.818 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.003696490, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.000133898, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.003429993, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000088281, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 7, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.006606204, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000943743 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.819 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "cct": { 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "total_workers": 1, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "unhealthy_workers": 0 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "finisher-MDSRank": { 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "queue_len": 0, 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "complete_latency": { 2024-09-19T09:42:19.820 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 3496, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.082183840, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000023507 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "queue_len": 0, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "complete_latency": { 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.821 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": { 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "request": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "reply": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "reply_latency": { 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "slow_reply": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "forward": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_commit": 0, 2024-09-19T09:42:19.822 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_split": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_merge": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes": 1829, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_top": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_bottom": 1829, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_pinned": 1829, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_expired": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "subtrees": 1, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_hit": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_forward": 0, 2024-09-19T09:42:19.823 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_discover": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "traverse_lock": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "load_cent": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "q": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "exported": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "exported_inodes": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "imported": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "imported_inodes": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:19.824 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rfiles": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rbytes": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "root_rsnaps": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:19.825 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_caps": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_cap_release": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "process_request_cap_release": 0 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache": { 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_update": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:19.826 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_try_discover": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_send_discover": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_created": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_enqueued": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "strays_migrated": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_started": 0, 2024-09-19T09:42:19.827 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_completed": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_flush": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "ireq_inodestats": 0 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log": { 2024-09-19T09:42:19.828 INFO:teuthology.orchestra.run.smithi028.stdout: "evlrg": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "evadd": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "evex": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "evtrm": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "ev": 9760, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "evexg": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "evexd": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segadd": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segex": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segtrm": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "seg": 12, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segmjr": 2, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segexg": 0, 2024-09-19T09:42:19.829 INFO:teuthology.orchestra.run.smithi028.stdout: "segexd": 0, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "expos": 4194304, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "wrpos": 18370721, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "rdpos": 18370122, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "jlat": { 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "replayed": 9760 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_mem": { 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 13, 2024-09-19T09:42:19.830 INFO:teuthology.orchestra.run.smithi028.stdout: "ino+": 1862, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "ino-": 1849, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dir": 11, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dir+": 99, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dir-": 88, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dn": 1829, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dn+": 4188, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "dn-": 2359, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "cap": 0, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "cap+": 0, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "cap-": 0, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "rss": 53356, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "heap": 215328 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.831 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_server": { 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "dispatch_client_request": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "dispatch_server_request": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_request": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_client_session": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "handle_peer_request": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "req_create_latency": { 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.832 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getattr_latency": { 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "req_link_latency": { 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.833 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookup_latency": { 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.834 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupino_latency": { 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupname_latency": { 2024-09-19T09:42:19.835 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: "req_lssnap_latency": { 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.837 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mkdir_latency": { 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mknod_latency": { 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "req_mksnap_latency": { 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.838 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "req_open_latency": { 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "req_readdir_latency": { 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rename_latency": { 2024-09-19T09:42:19.839 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmdir_latency": { 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:19.841 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setattr_latency": { 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.842 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setlayout_latency": { 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "req_setxattr_latency": { 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "req_symlink_latency": { 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "req_unlink_latency": { 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.844 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_sessions": { 2024-09-19T09:42:19.845 INFO:teuthology.orchestra.run.smithi028.stdout: "session_count": 1, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "session_add": 2, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "session_remove": 1, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "sessions_open": 1, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "sessions_stale": 0, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "total_load": 0, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "average_load": 0, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "avg_session_uptime": 1606, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "mempool": { 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:19.846 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:19.847 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:19.848 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_bytes": 197503, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_items": 55, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:19.849 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bytes": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_items": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_items": 152, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_items": 0, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_bytes": 1331208, 2024-09-19T09:42:19.850 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_items": 7444, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_items": 0, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_items": 0 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter": { 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_active": 0, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_laggy": 0, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_send": 2102, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_send_bytes": 21, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_resend": 3, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_reply": 2102, 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "op_latency": { 2024-09-19T09:42:19.851 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 2102, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 3.255910369, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.001548958 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op_inflight": 0, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "oplen_avg": { 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 2102, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 2103 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op": 2102, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op_r": 2102, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op_w": 0, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op_rmw": 0, 2024-09-19T09:42:19.852 INFO:teuthology.orchestra.run.smithi028.stdout: "op_pg": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_stat": 1280, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_create": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_read": 820, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_write": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_writefull": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_writesame": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_append": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_zero": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_truncate": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_delete": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_mapext": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:19.853 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_getxattr": 1, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_setxattr": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_call": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_watch": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_notify": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_pgls": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "osdop_other": 0, 2024-09-19T09:42:19.854 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_active": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_send": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_resend": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "linger_ping": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_active": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_send": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolop_resend": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_active": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_send": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "poolstat_resend": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_active": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_send": 0, 2024-09-19T09:42:19.855 INFO:teuthology.orchestra.run.smithi028.stdout: "statfs_resend": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "command_active": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "command_send": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "command_resend": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "map_epoch": 85, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "map_full": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "map_inc": 5, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_sessions": 6, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_session_open": 6, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_session_close": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_laggy": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_wr": 0, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_rd": 2, 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_del": 0 2024-09-19T09:42:19.856 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "oft": { 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_objs": 0, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_updates": 0, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "omap_total_removes": 0 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "osd-slow-ops": { 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "slow_ops_count": 0 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "purge_queue": { 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:19.857 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_executed": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 2331, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 15453591, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 2331, 2024-09-19T09:42:19.858 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2331, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 15453591, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:19.859 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 2102, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 14176517, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2102, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 14176517, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.860 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 1024, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 2102, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 2102, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 2102, 2024-09-19T09:42:19.861 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 2102, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 3758096384, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:19.862 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-write_buf_throttle-0x55e47a4ac6e0": { 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:19.863 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 3758096384, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:19.864 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.865 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:19.865 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:20.683 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T09:42:20.683 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T09:42:20.683 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-19T09:42:20.683 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-19T09:42:20.683 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-19T09:42:20.684 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T09:42:20.684 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T09:42:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:21.455 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-19T09:42:21.456 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:21.456 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 203, 2024-09-19T09:42:21.456 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 198, 2024-09-19T09:42:21.456 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 751984, 2024-09-19T09:42:21.456 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 121504, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 0, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.035186127, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.014221648, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.025957672, 2024-09-19T09:42:21.457 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:21.458 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:21.458 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 197, 2024-09-19T09:42:21.458 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.012091038, 2024-09-19T09:42:21.458 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000061375 2024-09-19T09:42:21.458 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 751984, 2024-09-19T09:42:21.459 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 121504 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 5, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 152, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 3232, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 105424, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 3, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 2, 2024-09-19T09:42:21.460 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.014137845, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.009196471, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.003171514, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 151, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.011372918, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000075317 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:21.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 105424 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_messages": 0, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages": 2, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_bytes": 0, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_bytes": 0, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_created_connections": 2, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_active_connections": 1, 2024-09-19T09:42:21.462 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_total_time": 0.000461236, 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_recv_time": 0.000368931, 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:21.463 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:21.464 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "cct": { 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "total_workers": 0, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "unhealthy_workers": 0 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "mempool": { 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:21.465 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:21.466 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:21.467 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_items": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_bytes": 150015, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_anon_items": 31, 2024-09-19T09:42:21.468 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:21.469 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "pgmap_items": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_bytes": 928, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_co_items": 2, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_1_items": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "unittest_2_items": 0 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "val": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "max": 104857600, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "get_started": 0, 2024-09-19T09:42:21.470 INFO:teuthology.orchestra.run.smithi028.stdout: "get": 204, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "get_sum": 723301, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "get_or_fail_success": 204, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "take": 0, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "take_sum": 0, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "put": 204, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "put_sum": 723301, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "wait": { 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "avgcount": 0, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "sum": 0.000000000, 2024-09-19T09:42:21.471 INFO:teuthology.orchestra.run.smithi028.stdout: "avgtime": 0.000000000 2024-09-19T09:42:21.472 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:21.472 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-19T09:42:21.472 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-19T09:42:21.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:21 smithi078 ceph-mon[35619]: pgmap v631: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 559 KiB/s wr, 187 op/s 2024-09-19T09:42:21.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:21 smithi038 ceph-mon[35271]: pgmap v631: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 559 KiB/s wr, 187 op/s 2024-09-19T09:42:21.763 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:21 smithi028 ceph-mon[30972]: pgmap v631: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 559 KiB/s wr, 187 op/s 2024-09-19T09:42:22.114 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T09:42:22.114 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T09:42:22.114 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T09:42:22.114 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T09:42:22.114 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-19T09:42:22.114 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T09:42:22.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T09:42:22.378 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:42:22.845 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T09:42:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 20637, 2024-09-19T09:42:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 17368, 2024-09-19T09:42:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 8902653, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 16814642, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 12, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 5, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 1.238838946, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.470085749, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.937985485, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.048678926, 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 17366, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.517673003, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000029809 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 34160, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.006209220, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000181 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 37206, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 45565, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 13223470, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 17966668, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 11, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 7, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 1.806958975, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.725632490, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.908826305, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.018639108, 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 45564, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 2.122050605, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000046572 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 68910, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.014082461, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000204 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 937056, 2024-09-19T09:42:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 126560 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1532, 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1624, 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 637547, 2024-09-19T09:42:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 1705264, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 18, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 8, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.124924812, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.049760553, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.058460332, 2024-09-19T09:42:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.018864439, 2024-09-19T09:42:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1623, 2024-09-19T09:42:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.068332187, 2024-09-19T09:42:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000042102 2024-09-19T09:42:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1568, 2024-09-19T09:42:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000279470, 2024-09-19T09:42:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000178 2024-09-19T09:42:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 336992 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1831, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3.418515422, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001867020 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3141, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.209362485, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000066654 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 18823, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 10256, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 10256, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 146.551697240, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.014289362 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 8567, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 46, 2024-09-19T09:42:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 4, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 24, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 6, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 8, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 10, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 13, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 408, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 1, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 1, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 3, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 32856, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 20642, 2024-09-19T09:42:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 8567, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 225, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 224, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 33, 2024-09-19T09:42:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 4, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 5, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 935, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 247, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 4, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 234, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 247, 2024-09-19T09:42:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 247, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 15, 2024-09-19T09:42:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 4916, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 5447, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 6915, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 1560, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 252, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 5810 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 735, 2024-09-19T09:42:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 3415, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 2030, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 2348, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 639, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 1563, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 1563, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 8, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 4393, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 33, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 4, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 8664, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 8664, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T09:42:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 10, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 10, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 16759410, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 910, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 15.002495382, 2024-09-19T09:42:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.016486258 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 28, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 1793, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 1765, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 27, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1694, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 1667, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 24, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 3759, 2024-09-19T09:42:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 3735, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 1, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 2519, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 2518, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 73380, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 223520 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 22451, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 178, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 18823, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 12, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 3485, 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T09:42:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1108, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 5.747397723, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005187182 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6191, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 102.502615872, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.016556713 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000117184, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000117184 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T09:42:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 950, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.522009294, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000549483 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 33, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.032419528, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000982409 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1110, 2024-09-19T09:42:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 8.180278665, 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.007369620 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T09:42:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T09:42:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 863, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 29.566858974, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.034260555 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 1463, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 1463, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 568, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 303591, 2024-09-19T09:42:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 109, 2024-09-19T09:42:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T09:42:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T09:42:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 152, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 115509, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 481, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T09:42:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3172, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 13589981, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 32, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3172, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3172, 2024-09-19T09:42:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 47.837011901, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.015081025 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3172, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3406 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3172, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 325, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 2847, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T09:42:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 10, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 78, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 988, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 78, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 4, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 1577, 2024-09-19T09:42:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 247, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 20, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T09:42:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T09:42:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 85, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 5, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 12, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 12, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-09-19T09:42:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 334, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 70 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 4366 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 6, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 1563, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 1563, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 59371, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 17403664, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 59371, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 59371, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 17403664, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3172, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 13747844, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3172, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 13747844, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3172, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3172, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3172, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3172, 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1563, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 157863, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1563, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 78, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 157863, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55d9f86ae500": { 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 8664, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 12565106, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 8664, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 910, 2024-09-19T09:42:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 12565106, 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:22.906 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T09:42:23.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:23 smithi078 ceph-mon[35619]: pgmap v632: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 395 KiB/s wr, 72 op/s 2024-09-19T09:42:23.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:23 smithi038 ceph-mon[35271]: pgmap v632: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 395 KiB/s wr, 72 op/s 2024-09-19T09:42:23.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:23 smithi028 ceph-mon[30972]: pgmap v632: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 395 KiB/s wr, 72 op/s 2024-09-19T09:42:23.868 INFO:tasks.check_counter:Daemon mds.b mds.exported=33 2024-09-19T09:42:23.868 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2024-09-19T09:42:23.868 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=735 2024-09-19T09:42:23.868 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3415 2024-09-19T09:42:23.868 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=4 2024-09-19T09:42:23.868 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T09:42:23.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T09:42:24.118 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:42:24.611 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 920, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 894, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1140752, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 341042, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.099240530, 2024-09-19T09:42:24.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.039569787, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.042870821, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.018109758, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 893, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.042972490, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000048121 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 909920, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 122576 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 202, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 348, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 51758, 2024-09-19T09:42:24.614 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 400244, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.026010343, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.009930855, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.007022413, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.006370451, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 347, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.021610562, 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000062278 2024-09-19T09:42:24.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 337536 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:24.616 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1236, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1238, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 12881578, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 393055, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.085365678, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.019216005, 2024-09-19T09:42:24.617 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.032291832, 2024-09-19T09:42:24.620 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.028397589, 2024-09-19T09:42:24.620 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:24.620 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1236, 2024-09-19T09:42:24.620 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.031433742, 2024-09-19T09:42:24.620 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000025431 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T09:42:24.621 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3570, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.079469163, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000022260 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.622 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-09-19T09:42:24.623 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-09-19T09:42:24.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 1572, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 1572, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 1572, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-09-19T09:42:24.625 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-09-19T09:42:24.626 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:24.628 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T09:42:24.629 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T09:42:24.630 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T09:42:24.631 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 8664, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T09:42:24.632 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 10, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 16759410, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 16758272, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.633 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 8664 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 12, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 1599, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 1587, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 10, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 67, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 57, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 1572, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 3325, 2024-09-19T09:42:24.634 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 1753, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 52452, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215328 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-09-19T09:42:24.635 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.638 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:24.639 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.641 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.644 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.645 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.646 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.648 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T09:42:24.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 1824, 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:24.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:24.652 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:24.653 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T09:42:24.654 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 190167, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 52, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:24.655 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 152, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 1146776, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 6405, 2024-09-19T09:42:24.656 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 2120, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 2120, 2024-09-19T09:42:24.657 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2120, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 4.331910538, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002043354 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2120, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 2121 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 2120, 2024-09-19T09:42:24.658 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 2120, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1290, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 828, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T09:42:24.659 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:24.660 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T09:42:24.661 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T09:42:24.662 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 85, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 5, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 5, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 5, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T09:42:24.663 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:24.664 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 2354, 2024-09-19T09:42:24.665 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 13847317, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 2354, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2354, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 13847317, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.666 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 2120, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 12565458, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2120, 2024-09-19T09:42:24.667 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 12565458, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T09:42:24.668 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 2120, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 2120, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2120, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 2120, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:24.670 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55a910ce66e0": { 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:24.671 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:24.672 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:24.673 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:24.673 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:24.673 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.673 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:24.673 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T09:42:25.358 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T09:42:25.358 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T09:42:25.358 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T09:42:25.358 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T09:42:25.358 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-19T09:42:25.358 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T09:42:25.358 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T09:42:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:42:25.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:25 smithi078 ceph-mon[35619]: pgmap v633: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 53 op/s 2024-09-19T09:42:25.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:25 smithi038 ceph-mon[35271]: pgmap v633: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 53 op/s 2024-09-19T09:42:25.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:25 smithi028 ceph-mon[30972]: pgmap v633: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 53 op/s 2024-09-19T09:42:26.204 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T09:42:26.204 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:26.204 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 28433, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 27805, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 11812512, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 11891590, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 9, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 5, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 1.630198772, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.636046856, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.249881902, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.030937661, 2024-09-19T09:42:26.205 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 27805, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.708311463, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000025474 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 52589, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.008391271, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000159 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 925808, 2024-09-19T09:42:26.206 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 182880 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 47291, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 57054, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 15441361, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 33443996, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 11, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 1.904765874, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.829136103, 2024-09-19T09:42:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 2.130522997, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.017304874, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 57052, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3.174487791, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000055642 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88327, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.015380308, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000174 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.208 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 337744 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2762, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2814, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 789066, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 11145162, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 22, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 9, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.221686415, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.084982572, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.098006187, 2024-09-19T09:42:26.209 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.049125638, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2812, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.110754459, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000039386 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1580, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000264881, 2024-09-19T09:42:26.210 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000167 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2411, 2024-09-19T09:42:26.211 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3.865907951, 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001603445 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3644, 2024-09-19T09:42:26.212 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.208068530, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000057098 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 26864, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 26856, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 26856, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 990.701572692, 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.036889394 2024-09-19T09:42:26.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 8, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 64, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 6, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 24, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 6, 2024-09-19T09:42:26.214 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 8, 2024-09-19T09:42:26.215 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 10, 2024-09-19T09:42:26.215 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 13, 2024-09-19T09:42:26.215 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 570, 2024-09-19T09:42:26.215 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 1, 2024-09-19T09:42:26.215 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 1, 2024-09-19T09:42:26.216 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 3, 2024-09-19T09:42:26.216 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 57727, 2024-09-19T09:42:26.216 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 43342, 2024-09-19T09:42:26.216 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 8, 2024-09-19T09:42:26.216 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 231, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 404, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 49, 2024-09-19T09:42:26.217 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 85, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 7, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 1427, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 213, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 4, 2024-09-19T09:42:26.218 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 234, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 213, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:26.219 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 213, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 190, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 6896, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 6883, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 10020, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 1735, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 160, 2024-09-19T09:42:26.220 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 6528 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 873, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 3510, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 2046, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 2314, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 780, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 1819, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 1819, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 8, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 4874, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 49, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 6, 2024-09-19T09:42:26.222 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 9760, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 9760, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T09:42:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 12, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 12, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 18370721, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1165, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 7.688524144, 2024-09-19T09:42:26.224 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.006599591 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 28, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 2071, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 2043, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 27, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1812, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 1785, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 24, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 4647, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 4623, 2024-09-19T09:42:26.225 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 1, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 3010, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 3009, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 79968, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 223520 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 31536, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 1413, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 26864, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 12, 2024-09-19T09:42:26.226 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 6453, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1170, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 6.992770657, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005976727 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.227 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.228 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 22169, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 939.750018947, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.042390275 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T09:42:26.229 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000169276, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000169276 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.230 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:26.231 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.231 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.231 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.231 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.232 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1151, 2024-09-19T09:42:26.233 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.891074852, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000774174 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 49, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.048902265, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000998005 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1173, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 41.251015253, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.035167105 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:26.234 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.235 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.237 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.238 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1143, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.767621442, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001546475 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.239 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T09:42:26.240 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 2771, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 2771, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 572, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:26.241 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:26.242 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:26.243 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 290599, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 122, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 152, 2024-09-19T09:42:26.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 116549, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 527, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T09:42:26.245 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3710, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 15496340, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 33, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3710, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3710, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 27.938424015, 2024-09-19T09:42:26.246 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.007530572 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3710, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3962 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3710, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 309, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 3401, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 10, 2024-09-19T09:42:26.247 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 96, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 1261, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 81, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 6, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 1834, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 213, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 20, 2024-09-19T09:42:26.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T09:42:26.249 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T09:42:26.250 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 85, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 5, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 12, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 12, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 367, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 74 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-09-19T09:42:26.251 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 4863 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 6, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 1819, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 1819, 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:26.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 78482, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 20962890, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 78482, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 78482, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 20962890, 2024-09-19T09:42:26.253 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:26.254 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3710, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 15680059, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3710, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 15680059, 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:26.256 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:26.257 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3710, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3710, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3710, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3710, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.258 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1819, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 183719, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1819, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 96, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 183719, 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:26.259 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x5651b239a500": { 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:26.260 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 9760, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 14176417, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 9760, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1165, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 14176417, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:26.261 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.262 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:26.262 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T09:42:27.051 INFO:tasks.check_counter:Daemon mds.h mds.exported=49 2024-09-19T09:42:27.051 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-09-19T09:42:27.051 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=873 2024-09-19T09:42:27.051 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3510 2024-09-19T09:42:27.051 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=6 2024-09-19T09:42:27.051 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T09:42:27.052 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T09:42:27.300 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:42:27.577 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:27 smithi038 ceph-mon[35271]: pgmap v634: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 19 op/s 2024-09-19T09:42:27.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:27 smithi078 ceph-mon[35619]: pgmap v634: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 19 op/s 2024-09-19T09:42:27.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:27 smithi028 ceph-mon[30972]: pgmap v634: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.0 KiB/s wr, 19 op/s 2024-09-19T09:42:27.797 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T09:42:27.797 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 241, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 213, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 914642, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 125268, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.036069366, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.012888458, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.030814976, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000018966, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 213, 2024-09-19T09:42:27.798 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.015235854, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000071529 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 913808, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 124944 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 600, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 751, 2024-09-19T09:42:27.799 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 147536, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 529719, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.059452398, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.021636700, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.012746221, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.018906915, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 749, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.028460614, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000037998 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:27.800 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 339232 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1370, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1372, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 411269, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 435718, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:27.801 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.126251583, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.044854004, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.032200922, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.037655088, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1370, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.052484354, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000038309 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3284, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.085435692, 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000026015 2024-09-19T09:42:27.803 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-09-19T09:42:27.804 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T09:42:27.805 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.808 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-09-19T09:42:27.809 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-09-19T09:42:27.810 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-09-19T09:42:27.811 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:27.812 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T09:42:27.813 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T09:42:27.814 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T09:42:27.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 7, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-09-19T09:42:27.816 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4196268, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 4196127, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.817 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 7 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 3, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 3, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 1, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-09-19T09:42:27.818 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 48712, 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215328 2024-09-19T09:42:27.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.820 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.821 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T09:42:27.824 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.825 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.825 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.826 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.827 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:27.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.829 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:27.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 3, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 2, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 2, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-09-19T09:42:27.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 3064, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:27.834 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:27.835 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T09:42:27.836 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 189423, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 51, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:27.837 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 152, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 8512, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 13, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T09:42:27.838 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 1970, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 1970, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1970, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 13.484721452, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.006845036 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1970, 2024-09-19T09:42:27.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1971 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 1970, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 1970, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1278, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-09-19T09:42:27.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 690, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-09-19T09:42:27.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T09:42:27.842 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 85, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 5, 2024-09-19T09:42:27.843 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 5, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 5, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.844 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:27.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 2207, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1259750, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 2207, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2207, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1259750, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1970, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1969, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1970, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1969, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.847 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1970, 2024-09-19T09:42:27.848 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1970, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1970, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1970, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:27.849 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:27.850 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55f9badca6e0": { 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T09:42:27.851 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T09:42:27.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T09:42:27.853 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.853 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T09:42:27.853 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T09:42:28.552 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T09:42:28.552 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T09:42:28.552 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T09:42:28.552 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T09:42:28.552 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-19T09:42:28.552 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T09:42:28.553 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T09:42:28.878 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:42:29.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:29 smithi078 ceph-mon[35619]: pgmap v635: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:42:29.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:29 smithi038 ceph-mon[35271]: pgmap v635: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:42:29.702 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-19T09:42:29.702 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:29.702 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 9857, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 9733, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3708586, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1176018, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 12, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.476159762, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.172924924, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.421534314, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000411351, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9732, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.156237982, 2024-09-19T09:42:29.703 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000016054 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 18545, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003038929, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000163 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 935568, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 128816 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 577, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 499, 2024-09-19T09:42:29.704 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 262265, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 549618, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 14, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.035119130, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.014558473, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.023132220, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000420973, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 497, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.025518878, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000051345 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.705 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 867, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000120957, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000139 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 339440 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 15278, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 8031, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 7082310, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1691618, 2024-09-19T09:42:29.706 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 13, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.777994260, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.343978583, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.619829796, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.003719632, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8030, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.173136669, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000021561 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:29.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 22971, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003519146, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000153 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:29.708 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 50, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001847264, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000036945 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000013742, 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000006871 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 9272, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 2, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000421344, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000210672 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 9270, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 11, 2024-09-19T09:42:29.710 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 25, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 7, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 18, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 13, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 400, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-19T09:42:29.711 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 3, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 11664, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 2151, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 9268, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 238, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-09-19T09:42:29.712 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:29.713 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 49, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 48, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 96, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 2, 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 2 2024-09-19T09:42:29.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 3941, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 2324, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 2882, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-19T09:42:29.715 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 8, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 678, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-19T09:42:29.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 7, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 7, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 2, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-09-19T09:42:29.717 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 4196268, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012090909, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002015151 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-09-19T09:42:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 29, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 229, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 200, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 28, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1911, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 1883, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 25, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 425, 2024-09-19T09:42:29.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 400, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 3, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 3, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 56992, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 223520 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 9305, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 17, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 9272, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 16, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 34, 2024-09-19T09:42:29.720 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000158437, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000158437 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.721 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000262907, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000262907 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.722 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-19T09:42:29.723 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.725 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-19T09:42:29.725 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.725 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.725 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.726 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.727 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.728 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.733 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 2, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 1, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 1, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 597, 2024-09-19T09:42:29.734 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:29.736 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:29.737 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 255623, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 94, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 152, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:29.738 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 118581, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 430, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-19T09:42:29.739 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 47, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 13767, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 33, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 47, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 47, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.211996687, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004510567 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 47, 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 75 2024-09-19T09:42:29.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 47, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 0, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 47, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 10, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 0, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 6, 2024-09-19T09:42:29.741 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 7, 2024-09-19T09:42:29.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-19T09:42:29.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-19T09:42:29.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-19T09:42:29.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-09-19T09:42:29.744 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 6, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 20, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:29.745 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-19T09:42:29.746 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 85, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 5, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 8, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 8, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-19T09:42:29.747 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 22, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 0, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 4 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 650 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.748 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:29.749 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 25708, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 8722447, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 25708, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 25708, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 8722447, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 47, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 13767, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 47, 2024-09-19T09:42:29.751 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 13767, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:29.752 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 47, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 47, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 47, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 47, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:29.753 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:29.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55869aa16500": { 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 7, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1964, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 7, 2024-09-19T09:42:29.755 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1964, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:29.756 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-19T09:42:29.764 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:29 smithi028 ceph-mon[30972]: pgmap v635: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:42:30.525 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T09:42:30.526 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T09:42:30.526 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T09:42:30.526 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=3941 2024-09-19T09:42:30.526 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-19T09:42:30.526 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T09:42:30.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T09:42:30.777 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:42:31.230 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 18085, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 12436, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 8920506, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 7045555, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 9, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 1.014824122, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.449570667, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.709767283, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.023007483, 2024-09-19T09:42:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 12437, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.368767740, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000029650 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 28075, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004340721, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000154 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 929376, 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 154496 2024-09-19T09:42:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 947, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1039, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 382718, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 786790, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 13, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.062414244, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.024710502, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.036538362, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004313748, 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1038, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.061975505, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000059706 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1408, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000217804, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000154 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 341072 2024-09-19T09:42:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 19051, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 21664, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 5894899, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 8063769, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 20, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 10, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.829828315, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.330373204, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.683785052, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.007325004, 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 21660, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.561252464, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000025911 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 36436, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.006179205, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000169 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3376, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 4528 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-19T09:42:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 934, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.200605171, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001285444 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1012, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.056307458, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000055639 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 14150, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 5000, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5000, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 75.776584065, 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.015155316 2024-09-19T09:42:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 9150, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 1, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 27, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 2, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 24, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 6, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 8, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 10, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 13, 2024-09-19T09:42:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 438, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 3, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 21878, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 11369, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 9149, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 242, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 43, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-19T09:42:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 16, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 19, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 2, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 418, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 32, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 32, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 32, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 7, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 1541, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 1633, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 2138, 2024-09-19T09:42:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 496, 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 58, 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 1739 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 333, 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 3758, 2024-09-19T09:42:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 2209, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 2676, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 207, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 496, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 496, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 8, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 2280, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 16, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 2, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-19T09:42:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 2852, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 2852, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-19T09:42:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 4, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 4, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 8110088, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 482, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 15.965946869, 2024-09-19T09:42:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.033124371 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 28, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 731, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 703, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 27, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1856, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 1829, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 24, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 1635, 2024-09-19T09:42:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 1611, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 870, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 870, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 64504, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 223520 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 15328, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 110, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 14150, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 11, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 1175, 2024-09-19T09:42:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 288, 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.107095238, 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.007316302 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2024-09-19T09:42:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001654489, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001654489 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3821, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 71.585254203, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.018734690 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-19T09:42:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 234, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.096983428, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000414459 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 16, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.013204097, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000825256 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 288, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.469303059, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.005101746 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-19T09:42:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 352, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.503089551, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001429231 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 2, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 876, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 876, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 492, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 295687, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 108, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 152, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 115197, 2024-09-19T09:42:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 474, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 1286, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 4221849, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 33, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 1286, 2024-09-19T09:42:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1286, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 29.982540222, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.023314572 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1286, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1436 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 1286, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 69, 2024-09-19T09:42:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 1217, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 10, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 36, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 518, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 54, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 2, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 508, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 32, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 20, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-19T09:42:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 85, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 5, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 209, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 45 2024-09-19T09:42:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 2252 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 6, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 496, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 496, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 38079, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 11753903, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 38079, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 38079, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 11753903, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 1286, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 4271929, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1286, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 4271929, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 1286, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1286, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1286, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1286, 2024-09-19T09:42:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 496, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 50096, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 496, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 36, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 50096, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55834807e500": { 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 2852, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 3915784, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 2852, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 482, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3915784, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:31.281 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-19T09:42:31.539 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:31 smithi078 ceph-mon[35619]: pgmap v636: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:42:31.584 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:31 smithi028 ceph-mon[30972]: pgmap v636: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:42:31.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:31 smithi038 ceph-mon[35271]: pgmap v636: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:42:32.028 INFO:tasks.check_counter:Daemon mds.f mds.exported=16 2024-09-19T09:42:32.028 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-09-19T09:42:32.028 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=333 2024-09-19T09:42:32.028 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=3758 2024-09-19T09:42:32.028 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=2 2024-09-19T09:42:32.028 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T09:42:32.028 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T09:42:32.275 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 86789, 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 93129, 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 24943709, 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 317484785, 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 55, 2024-09-19T09:42:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 6.595251064, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 2.827286471, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 1.983286527, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 1.813131448, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 93128, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 43.265046823, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000464576 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 78815, 2024-09-19T09:42:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.013587898, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000172 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 69454, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 76466, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 25665252, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 276177191, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 10, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 6, 2024-09-19T09:42:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 5.945330526, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 2.511535896, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 1.255127301, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 1.976342130, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 76468, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 62.757560948, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000820703 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 22110, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.005581926, 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000252 2024-09-19T09:42:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 944144, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 133824 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 483588, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 730823, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 156240121, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 576116418, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 112, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 9, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 22.413386216, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 9.825263372, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 36.154814805, 2024-09-19T09:42:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 1.677357331, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 730818, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 55.089651013, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000075380 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 977034, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.153731217, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000157 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 92000, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 378080 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 31967, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 255.743517212, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.008000235 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 88940, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 31.686286916, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000356265 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 258767, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 255548, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 255548, 2024-09-19T09:42:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 10569.055841740, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.041358397 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 3219, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 28, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 1, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 11320, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 157, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 13, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 153, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 76, 2024-09-19T09:42:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 62, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 15, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 44, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 4538, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 1, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 1, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 5, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 419678, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 360483, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 3209, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 95, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 4, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 293, 2024-09-19T09:42:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 14, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 2780, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 98, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 108, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 5125, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 5125, 2024-09-19T09:42:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 84, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 4, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 80, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 5041, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 40, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 112925, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 130524, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 8, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 168978, 2024-09-19T09:42:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 49644, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 9258, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 100917 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 4074, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 54, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 8648, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 3, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 48348, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 48345, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 8, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 102434, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 4, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 14, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-19T09:42:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 175, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 283956, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 280842, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 280842, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 3114, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 9073, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 377, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 373, 2024-09-19T09:42:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 373, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 4, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 1, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 13, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 632953068, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 639175627, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 21656, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1060.959792616, 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.048991493 2024-09-19T09:42:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 43, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 49958, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 49915, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 38, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1602, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 1564, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 153, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 113159, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 113006, 2024-09-19T09:42:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 1, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 86239, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 86238, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 198408, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 231712 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 270029, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 7718, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 258767, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 32, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 17063, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 49460, 2024-09-19T09:42:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 986.502956434, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.019945470 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6324, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1049.020247972, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.165879229 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-19T09:42:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 104771, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6753.035652427, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.064455198 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 207, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 9.148850346, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.044197344 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 14879, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 588.799776495, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.039572536 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1300, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.846810291, 2024-09-19T09:42:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002189854 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 49077, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1090.661296302, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.022223471 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 201, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 22.023374858, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.109569029 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.073326761, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.119258529 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-19T09:42:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.020819029, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002602378 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001863480, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000621160 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 29309, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 65.920867345, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002249168 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 2, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 45379, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 45379, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 605, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 305322, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 122, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 352, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 4, 2024-09-19T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 152, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 269038, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2606, 2024-09-19T09:42:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 141100, 2024-09-19T09:42:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 748844463, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 37, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 141100, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 141100, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 22954.030753927, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.162679169 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 141100, 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 266228 2024-09-19T09:42:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 141100, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 6000, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 135100, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 9846, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 49599, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 842, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 22495, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 519, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-19T09:42:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 157, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 50516, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 5138, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 99199, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-19T09:42:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 8, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-19T09:42:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 85, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 5, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-19T09:42:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 22614, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 58, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 5237 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 102501 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-19T09:42:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 14, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 6, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 49625, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 48345, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 639827, 2024-09-19T09:42:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 149237423, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 639827, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 639827, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 149237423, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 141100, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 753728176, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 141100, 2024-09-19T09:42:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 753728176, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 141100, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 141100, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 141100, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 141100, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 48345, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 4884165, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 48345, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 839, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 4884165, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x5614fd2c6500": { 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 283956, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 634981323, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 283956, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 21656, 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 634981323, 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:32.778 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-19T09:42:33.598 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:33 smithi078 ceph-mon[35619]: pgmap v637: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T09:42:33.600 INFO:tasks.check_counter:Daemon mds.i mds.exported=14 2024-09-19T09:42:33.600 INFO:tasks.check_counter:Daemon mds.i mds.imported=98 2024-09-19T09:42:33.600 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4074 2024-09-19T09:42:33.600 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-19T09:42:33.600 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=157 2024-09-19T09:42:33.600 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T09:42:33.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T09:42:33.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:33 smithi038 ceph-mon[35271]: pgmap v637: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T09:42:33.765 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:33 smithi028 ceph-mon[30972]: pgmap v637: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T09:42:33.849 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:42:34.387 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-19T09:42:34.387 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:42:34.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 88, 2024-09-19T09:42:34.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 88, 2024-09-19T09:42:34.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 5863378, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 27170, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 6, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.267901684, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.002495192, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.261649834, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.002057119, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 87, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.516287318, 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.005934336 2024-09-19T09:42:34.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 20, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 48, 2024-09-19T09:42:34.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 405914, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 108755, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.004776704, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.001354851, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.002558412, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000447125, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 46, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003769769, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000081951 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 102656 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 62, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 61, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 172177, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 28826, 2024-09-19T09:42:34.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.007225895, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.002399252, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.006761657, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000461395, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 60, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004171614, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000069526 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:42:34.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 167584, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 23728 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-19T09:42:34.393 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 180, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004658804, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000025882 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.394 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-09-19T09:42:34.395 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 1590, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 1590, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 1590, 2024-09-19T09:42:34.396 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 2, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-19T09:42:34.397 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-19T09:42:34.398 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:42:34.399 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-19T09:42:34.400 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:42:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-19T09:42:34.402 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 3114, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-19T09:42:34.403 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 4, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 1, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 632953068, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 639175627, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 639173975, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.404 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 3114 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 15, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 1532, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 1517, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 14, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 100, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 86, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 1590, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 3074, 2024-09-19T09:42:34.405 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 1484, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 49716, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215328 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-09-19T09:42:34.406 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-19T09:42:34.407 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.408 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-19T09:42:34.412 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.413 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.414 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.416 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.417 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.418 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.419 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-19T09:42:34.420 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1827, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-19T09:42:34.421 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:42:34.422 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:42:34.423 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 196670, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 53, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-19T09:42:34.424 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7400, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 152, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 1171777, 2024-09-19T09:42:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 6815, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 119, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 26, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 4, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 119, 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-19T09:42:34.426 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 119, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.075998895, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000638646 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 119, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 120 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 119, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 119, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-19T09:42:34.427 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 72, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 44, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-19T09:42:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 2, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-19T09:42:34.429 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-19T09:42:34.430 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 6, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 6, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.431 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:42:34.432 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 166, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 6421108, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:34.433 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 166, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 166, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6421108, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:34.435 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:34.436 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:34.436 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:34.436 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:34.436 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 119, 2024-09-19T09:42:34.437 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 6222569, 2024-09-19T09:42:34.437 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 119, 2024-09-19T09:42:34.437 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6222569, 2024-09-19T09:42:34.437 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:34.437 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:34.438 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 119, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 119, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 119, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 119, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:34.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:34.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x560c066ba6e0": { 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-19T09:42:34.441 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-19T09:42:34.444 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-19T09:42:34.444 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-19T09:42:34.444 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-19T09:42:34.445 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-19T09:42:35.177 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T09:42:35.178 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T09:42:35.178 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T09:42:35.178 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-19T09:42:35.178 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-19T09:42:35.178 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T09:42:35.188 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T09:42:35.188 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:42:35.188 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:42:35.211 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:42:35.211 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:42:35.272 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T09:42:35.583 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:35.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:35 smithi078 ceph-mon[35619]: pgmap v638: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:35.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:35 smithi038 ceph-mon[35271]: pgmap v638: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:35.704 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:35 smithi028 ceph-mon[30972]: pgmap v638: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.050+0000 7f04fc62b640 1 -- 172.21.15.28:0/669573061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4106f10 msgr2=0x7f04f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.050+0000 7f04fc62b640 1 --2- 172.21.15.28:0/669573061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4106f10 0x7f04f410ea90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009ca0 tx=0x7f04ec02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- 172.21.15.28:0/669573061 shutdown_connections 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- 172.21.15.28:0/669573061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4106f10 0x7f04f410ea90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- 172.21.15.28:0/669573061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4102790 0x7f04f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- 172.21.15.28:0/669573061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4101de0 0x7f04f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- 172.21.15.28:0/669573061 >> 172.21.15.28:0/669573061 conn(0x7f04f40fd8c0 msgr2=0x7f04f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- 172.21.15.28:0/669573061 shutdown_connections 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- 172.21.15.28:0/669573061 wait complete. 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 Processor -- start 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- start start 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:36.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 0x7f04f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 0x7f04f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4111f70 con 0x7f04f4101de0 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4111df0 con 0x7f04f4102790 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.051+0000 7f04fc62b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41120f0 con 0x7f04f4106f10 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33420/0 (socket says 172.21.15.28:33420) 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 -- 172.21.15.28:0/4249459967 learned_addr learned my addr 172.21.15.28:0/4249459967 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04f9bc6640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 0x7f04f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fabc8640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 0x7f04f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 -- 172.21.15.28:0/4249459967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 msgr2=0x7f04f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:36.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 0x7f04f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 -- 172.21.15.28:0/4249459967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 msgr2=0x7f04f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 0x7f04f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 -- 172.21.15.28:0/4249459967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f41a6850 con 0x7f04f4101de0 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04f9bc6640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 0x7f04f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fabc8640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 0x7f04f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fa3c7640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f04f000c920 tx=0x7f04f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0007c70 con 0x7f04f4101de0 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f41a6b40 con 0x7f04f4101de0 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0007e10 con 0x7f04f4101de0 2024-09-19T09:42:36.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0018890 con 0x7f04f4101de0 2024-09-19T09:42:36.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.052+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41a70d0 con 0x7f04f4101de0 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.053+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f41021c0 con 0x7f04f4101de0 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.054+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f04f0018a70 con 0x7f04f4101de0 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.055+0000 7f04eb7fe640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 0x7f04e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.055+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f04f00cfae0 con 0x7f04f4101de0 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.055+0000 7f04f9bc6640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 0x7f04e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:36.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.056+0000 7f04f9bc6640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 0x7f04e00a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005d90 tx=0x7f04e4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.059+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04f0014030 con 0x7f04f4101de0 2024-09-19T09:42:36.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.249+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04f41a7300 con 0x7f04f4101de0 2024-09-19T09:42:36.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.250+0000 7f04eb7fe640 1 -- 172.21.15.28:0/4249459967 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+590 (secure 0 0 0) 0x7f04f000de20 con 0x7f04f4101de0 2024-09-19T09:42:36.252 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/4138562507 2024-09-20T09:23:47.303309+0000 2024-09-19T09:42:36.252 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/3407590911 2024-09-20T09:23:47.303309+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/78423353 2024-09-20T09:23:47.303309+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/1486296346 2024-09-20T09:23:47.303309+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6801/1042365655 2024-09-20T09:23:22.243952+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/1042365655 2024-09-20T09:23:22.243952+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6800/3407590911 2024-09-20T09:23:47.303309+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2768039084 2024-09-20T09:23:22.243952+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/3190205246 2024-09-20T09:23:22.243952+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:0/2766061860 2024-09-20T09:23:22.243952+0000 2024-09-19T09:42:36.253 INFO:teuthology.orchestra.run.smithi028.stderr:listed 10 entries 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 msgr2=0x7f04e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 0x7f04e00a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005d90 tx=0x7f04e4005d20 comp rx=0 tx=0).stop 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 msgr2=0x7f04f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f04f000c920 tx=0x7f04f000cde0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 shutdown_connections 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f04e00a3240 0x7f04e00a5700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f4102790 0x7f04f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f4106f10 0x7f04f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.252+0000 7f04fc62b640 1 --2- 172.21.15.28:0/4249459967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04f4101de0 0x7f04f40744f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.253+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 >> 172.21.15.28:0/4249459967 conn(0x7f04f40fd8c0 msgr2=0x7f04f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.253+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 shutdown_connections 2024-09-19T09:42:36.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:36.253+0000 7f04fc62b640 1 -- 172.21.15.28:0/4249459967 wait complete. 2024-09-19T09:42:36.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:36 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4249459967' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:42:36.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:36 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4249459967' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:42:36.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:36 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4249459967' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:42:36.964 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T09:42:36.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:36.965 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T09:42:37.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:42:37.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.077 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T09:42:37.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.124 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-19T09:42:37.168 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T09:42:37.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:42:37.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.228 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-19T09:42:37.318 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T09:42:37.318 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T09:42:37.318 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:42:37.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17679 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a998eb4aec29afa1d0a67a0811a3901af8c46b2a51bf579d6e2663fdbfaf1f19/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/a998eb4aec29afa1d0a67a0811a3901af8c46b2a51bf579d6e2663fdbfaf1f19/diff,workdir=/var/lib/containers/storage/overlay/a998eb4aec29afa1d0a67a0811a3901af8c46b2a51bf579d6e2663fdbfaf1f19/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bed4ae9e12e940a84039e302ec932a50f29ed7a47a02e6a1b3f1571969338c66/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/bed4ae9e12e940a84039e302ec932a50f29ed7a47a02e6a1b3f1571969338c66/diff,workdir=/var/lib/containers/storage/overlay/bed4ae9e12e940a84039e302ec932a50f29ed7a47a02e6a1b3f1571969338c66/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/da4d084d94c210284845e0049bf06f729ade7370e4809c582760ec3cfb4a6085/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/da4d084d94c210284845e0049bf06f729ade7370e4809c582760ec3cfb4a6085/diff,workdir=/var/lib/containers/storage/overlay/da4d084d94c210284845e0049bf06f729ade7370e4809c582760ec3cfb4a6085/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dee94e78888185cbe0d898e39bb99978cfaa97c7cfd98ec7fc3848dee3114b60/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/dee94e78888185cbe0d898e39bb99978cfaa97c7cfd98ec7fc3848dee3114b60/diff,workdir=/var/lib/containers/storage/overlay/dee94e78888185cbe0d898e39bb99978cfaa97c7cfd98ec7fc3848dee3114b60/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f46c67d4f514da22fe942a7a6a6d1e28116fcd6b9a1704687cc56b44f7b4b80a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/f46c67d4f514da22fe942a7a6a6d1e28116fcd6b9a1704687cc56b44f7b4b80a/diff,workdir=/var/lib/containers/storage/overlay/f46c67d4f514da22fe942a7a6a6d1e28116fcd6b9a1704687cc56b44f7b4b80a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/944da6a4ca9fd31fb56c584b315ab59e80a9540d04e947473128317b0507758d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/944da6a4ca9fd31fb56c584b315ab59e80a9540d04e947473128317b0507758d/diff,workdir=/var/lib/containers/storage/overlay/944da6a4ca9fd31fb56c584b315ab59e80a9540d04e947473128317b0507758d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/df8e0aa916397d578c8ed27410aa69ef0afcae874e0424ac7154409b43ee4670/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/df8e0aa916397d578c8ed27410aa69ef0afcae874e0424ac7154409b43ee4670/diff,workdir=/var/lib/containers/storage/overlay/df8e0aa916397d578c8ed27410aa69ef0afcae874e0424ac7154409b43ee4670/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c6df1f4cf721496bc70842f0353275974e1439226573d983c88e52b613f2945e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/c6df1f4cf721496bc70842f0353275974e1439226573d983c88e52b613f2945e/diff,workdir=/var/lib/containers/storage/overlay/c6df1f4cf721496bc70842f0353275974e1439226573d983c88e52b613f2945e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0f122e0c1bbd8d132eb70793f5065d6245ce9d585a9f49c52144c841ca3026b2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/0f122e0c1bbd8d132eb70793f5065d6245ce9d585a9f49c52144c841ca3026b2/diff,workdir=/var/lib/containers/storage/overlay/0f122e0c1bbd8d132eb70793f5065d6245ce9d585a9f49c52144c841ca3026b2/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 overlay /var/lib/containers/storage/overlay/004bfd55abac1ea7681e0636c4cb45670b37b2cfa3e45c412227fb62fa7c8599/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CVHKWPLK4OZW72JYCWNGFSEBYO:/var/lib/containers/storage/overlay/l/XX73FR726UBFP5JBM2ZDTI2QEE,upperdir=/var/lib/containers/storage/overlay/004bfd55abac1ea7681e0636c4cb45670b37b2cfa3e45c412227fb62fa7c8599/diff,workdir=/var/lib/containers/storage/overlay/004bfd55abac1ea7681e0636c4cb45670b37b2cfa3e45c412227fb62fa7c8599/work,metacopy=on,volatile 0 0 2024-09-19T09:42:37.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.355 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-19T09:42:37.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:42:37.418 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-19T09:42:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-09-19T09:42:37.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:42:37.489 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T09:42:37.500 INFO:tasks.cephadm:Teardown begin 2024-09-19T09:42:37.500 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T09:42:37.501 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:42:37.553 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:37 smithi028 ceph-mon[30972]: pgmap v639: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:37.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:37 smithi078 ceph-mon[35619]: pgmap v639: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:37.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:37 smithi038 ceph-mon[35271]: pgmap v639: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:42:37.786 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/1287188942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c106a80 msgr2=0x7fe89c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/1287188942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c106a80 0x7fe89c106e60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009930 tx=0x7fe88c02a420 comp rx=0 tx=0).stop 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/1287188942 shutdown_connections 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/1287188942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c100a80 0x7fe89c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/1287188942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 0x7fe89c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/1287188942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c106a80 0x7fe89c106e60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/1287188942 >> 172.21.15.28:0/1287188942 conn(0x7fe89c0fc7a0 msgr2=0x7fe89c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/1287188942 shutdown_connections 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.226+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/1287188942 wait complete. 2024-09-19T09:42:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 Processor -- start 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 -- start start 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 0x7fe89c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c106a80 0x7fe89c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c110e80 con 0x7fe89c100a80 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c110d00 con 0x7fe89c106a80 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.227+0000 7fe8a0a9a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c111000 con 0x7fe89c101420 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33430/0 (socket says 172.21.15.28:33430) 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 -- 172.21.15.28:0/4105374780 learned_addr learned my addr 172.21.15.28:0/4105374780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89ad9d640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c106a80 0x7fe89c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe899d9b640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 0x7fe89c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 -- 172.21.15.28:0/4105374780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 msgr2=0x7fe89c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 0x7fe89c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 -- 172.21.15.28:0/4105374780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c106a80 msgr2=0x7fe89c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c106a80 0x7fe89c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 -- 172.21.15.28:0/4105374780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c1a4850 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe89a59c640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009230 tx=0x7fe88c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c03f030 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe88c003ad0 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c197080 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c035070 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.228+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c1975b0 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.229+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c100ee0 con 0x7fe89c100a80 2024-09-19T09:42:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.231+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe88c042030 con 0x7fe89c100a80 2024-09-19T09:42:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.231+0000 7fe88b7fe640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 0x7fe8840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.231+0000 7fe899d9b640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 0x7fe8840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.231+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7fe88c0f0100 con 0x7fe89c100a80 2024-09-19T09:42:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.232+0000 7fe899d9b640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 0x7fe8840a5780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe89c06b2e0 tx=0x7fe89000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.235+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe88c0bc800 con 0x7fe89c100a80 2024-09-19T09:42:38.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.450+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe89c108ff0 con 0x7fe89c100a80 2024-09-19T09:42:38.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.451+0000 7fe88b7fe640 1 -- 172.21.15.28:0/4105374780 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11965 (secure 0 0 0) 0x7fe88c0c0b00 con 0x7fe89c100a80 2024-09-19T09:42:38.453 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:38.453 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":49,"btime":"2024-09-19T09:42:05:685861+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":17043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/1897083288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":1897083288},{"type":"v1","addr":"172.21.15.28:6837","nonce":1897083288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:42:05.685838+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26392":{"gid":26392,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2606259516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2606259516},{"type":"v1","addr":"172.21.15.78:6833","nonce":2606259516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:42:38.454 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 49 2024-09-19T09:42:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 msgr2=0x7fe8840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 0x7fe8840a5780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe89c06b2e0 tx=0x7fe89000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 msgr2=0x7fe89c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009230 tx=0x7fe88c004270 comp rx=0 tx=0).stop 2024-09-19T09:42:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 shutdown_connections 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fe8840a32c0 0x7fe8840a5780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe89c106a80 0x7fe89c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe89c101420 0x7fe89c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 --2- 172.21.15.28:0/4105374780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe89c100a80 0x7fe89c199290 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 >> 172.21.15.28:0/4105374780 conn(0x7fe89c0fc7a0 msgr2=0x7fe89c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 shutdown_connections 2024-09-19T09:42:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:38.453+0000 7fe8a0a9a640 1 -- 172.21.15.28:0/4105374780 wait complete. 2024-09-19T09:42:39.054 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:42:39.353 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:39.600 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:39 smithi028 ceph-mon[30972]: pgmap v640: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:42:39.600 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:39 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4105374780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:39 smithi078 ceph-mon[35619]: pgmap v640: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:42:39.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:39 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4105374780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:39 smithi038 ceph-mon[35271]: pgmap v640: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T09:42:39.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:39 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4105374780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3868949757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 msgr2=0x7fcfb4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3868949757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb4106e60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009930 tx=0x7fcfac02a420 comp rx=0 tx=0).stop 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3868949757 shutdown_connections 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3868949757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 0x7fcfb4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3868949757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 0x7fcfb410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3868949757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb4106e60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3868949757 >> 172.21.15.28:0/3868949757 conn(0x7fcfb40fc7a0 msgr2=0x7fcfb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:39.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3868949757 shutdown_connections 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.789+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3868949757 wait complete. 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 Processor -- start 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 -- start start 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 0x7fcfb41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 0x7fcfb4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4110e80 con 0x7fcfb4106a80 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4110d00 con 0x7fcfb4100a80 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.790+0000 7fcfbb73a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4111000 con 0x7fcfb4101420 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33456/0 (socket says 172.21.15.28:33456) 2024-09-19T09:42:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 -- 172.21.15.28:0/3419447811 learned_addr learned my addr 172.21.15.28:0/3419447811 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb94d6640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 0x7fcfb41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb8cd5640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 0x7fcfb4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 -- 172.21.15.28:0/3419447811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 msgr2=0x7fcfb4197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 0x7fcfb4197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 -- 172.21.15.28:0/3419447811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 msgr2=0x7fcfb41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 0x7fcfb41972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 -- 172.21.15.28:0/3419447811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb419c270 con 0x7fcfb4106a80 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb9cd7640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400cce0 tx=0x7fcfa400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400f450 con 0x7fcfb4106a80 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa400fad0 con 0x7fcfb4106a80 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb419c560 con 0x7fcfb4106a80 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4018c10 con 0x7fcfb4106a80 2024-09-19T09:42:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.791+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a3e40 con 0x7fcfb4106a80 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.792+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb4100ee0 con 0x7fcfb4106a80 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.794+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcfa400f5f0 con 0x7fcfb4106a80 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.794+0000 7fcfb27fc640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 0x7fcf9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.794+0000 7fcfb94d6640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 0x7fcf9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.794+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7fcfa40cfe90 con 0x7fcfb4106a80 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.795+0000 7fcfb94d6640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 0x7fcf9c0a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:39.797+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcfa4014030 con 0x7fcfb4106a80 2024-09-19T09:42:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.010+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfb4198620 con 0x7fcfb4106a80 2024-09-19T09:42:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.011+0000 7fcfb27fc640 1 -- 172.21.15.28:0/3419447811 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11965 (secure 0 0 0) 0x7fcfb4198620 con 0x7fcfb4106a80 2024-09-19T09:42:40.013 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:40.013 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":49,"btime":"2024-09-19T09:42:05:685861+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4241327209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4241327209},{"type":"v1","addr":"172.21.15.28:6835","nonce":4241327209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":17043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/1897083288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":1897083288},{"type":"v1","addr":"172.21.15.28:6837","nonce":1897083288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:30:43.054690+0000","modified":"2024-09-19T09:42:05.685838+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24478,"mds_3":24472,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6839/1981224443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6838","nonce":1981224443},{"type":"v1","addr":"172.21.15.28:6839","nonce":1981224443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.28:6841/438294996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6840","nonce":438294996},{"type":"v1","addr":"172.21.15.28:6841","nonce":438294996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3013823435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3013823435},{"type":"v1","addr":"172.21.15.38:6833","nonce":3013823435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/347872214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":347872214},{"type":"v1","addr":"172.21.15.38:6835","nonce":347872214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":76,"addr":"172.21.15.38:6837/560457008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":560457008},{"type":"v1","addr":"172.21.15.38:6837","nonce":560457008}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":76,"addr":"172.21.15.78:6835/2086564778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2086564778},{"type":"v1","addr":"172.21.15.78:6835","nonce":2086564778}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.78:6837/2648819818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2648819818},{"type":"v1","addr":"172.21.15.78:6837","nonce":2648819818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6839/3114720816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3114720816},{"type":"v1","addr":"172.21.15.38:6839","nonce":3114720816}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.78:6839/3581897338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3581897338},{"type":"v1","addr":"172.21.15.78:6839","nonce":3581897338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26392":{"gid":26392,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/2606259516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2606259516},{"type":"v1","addr":"172.21.15.78:6833","nonce":2606259516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24455,24472,24478,24482,24499]},"id":1}]} 2024-09-19T09:42:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 49 2024-09-19T09:42:40.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 msgr2=0x7fcf9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:40.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 0x7fcf9c0a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac0058d0 comp rx=0 tx=0).stop 2024-09-19T09:42:40.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 msgr2=0x7fcfb419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400cce0 tx=0x7fcfa400a620 comp rx=0 tx=0).stop 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 shutdown_connections 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7fcf9c0a32c0 0x7fcf9c0a5780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcfb4106a80 0x7fcfb419bb90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb4101420 0x7fcfb4197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 --2- 172.21.15.28:0/3419447811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb4100a80 0x7fcfb41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 >> 172.21.15.28:0/3419447811 conn(0x7fcfb40fc7a0 msgr2=0x7fcfb40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 shutdown_connections 2024-09-19T09:42:40.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:40.013+0000 7fcfbb73a640 1 -- 172.21.15.28:0/3419447811 wait complete. 2024-09-19T09:42:40.557 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:40 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/3419447811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:40.559 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:42:40.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:40 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/3419447811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:40.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:40 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/3419447811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:42:40.813 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:41.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.246+0000 7f5c447f4640 1 -- 172.21.15.28:0/4235042855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c101420 msgr2=0x7f5c3c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:41.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.246+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4235042855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c101420 0x7f5c3c10d950 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009cd0 tx=0x7f5c3402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 -- 172.21.15.28:0/4235042855 shutdown_connections 2024-09-19T09:42:41.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4235042855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c101420 0x7f5c3c10d950 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4235042855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c100a80 0x7f5c3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4235042855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c106a80 0x7f5c3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 -- 172.21.15.28:0/4235042855 >> 172.21.15.28:0/4235042855 conn(0x7f5c3c0fc7a0 msgr2=0x7f5c3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 -- 172.21.15.28:0/4235042855 shutdown_connections 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 -- 172.21.15.28:0/4235042855 wait complete. 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 Processor -- start 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.247+0000 7f5c447f4640 1 -- start start 2024-09-19T09:42:41.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:41.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c101420 0x7f5c3c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:41.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c106a80 0x7f5c3c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:41.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c110e80 con 0x7f5c3c100a80 2024-09-19T09:42:41.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c110d00 con 0x7f5c3c101420 2024-09-19T09:42:41.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c447f4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c111000 con 0x7f5c3c106a80 2024-09-19T09:42:41.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:41.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33472/0 (socket says 172.21.15.28:33472) 2024-09-19T09:42:41.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 -- 172.21.15.28:0/4165317479 learned_addr learned my addr 172.21.15.28:0/4165317479 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:41.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42d91640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c106a80 0x7f5c3c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:41.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c41d8f640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c101420 0x7f5c3c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 -- 172.21.15.28:0/4165317479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c106a80 msgr2=0x7f5c3c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c106a80 0x7f5c3c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 -- 172.21.15.28:0/4165317479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c101420 msgr2=0x7f5c3c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c101420 0x7f5c3c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 -- 172.21.15.28:0/4165317479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c19d3d0 con 0x7f5c3c100a80 2024-09-19T09:42:41.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.248+0000 7f5c42590640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9f0 tx=0x7f5c3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.249+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800cd80 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.249+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38010910 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.249+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c19d6c0 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.249+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38018990 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.249+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c1a4fa0 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.250+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c100ee0 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.251+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5c38010ac0 con 0x7f5c3c100a80 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.252+0000 7f5c337fe640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 0x7f5c280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.252+0000 7f5c41d8f640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 0x7f5c280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.252+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f5c380d0080 con 0x7f5c3c100a80 2024-09-19T09:42:41.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.252+0000 7f5c41d8f640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 0x7f5c280a5700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005fd0 tx=0x7f5c2c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:41.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.255+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c3809cdd0 con 0x7f5c3c100a80 2024-09-19T09:42:41.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.434+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c3c1a5230 con 0x7f5c3c100a80 2024-09-19T09:42:41.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.436+0000 7f5c337fe640 1 -- 172.21.15.28:0/4165317479 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22580 (secure 0 0 0) 0x7f5c3809c6d0 con 0x7f5c3c100a80 2024-09-19T09:42:41.439 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T09:42:41.439 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":85,"fsid":"9a2e9aa4-7668-11ef-bceb-c7b262605968","created":"2024-09-19T09:22:57.723753+0000","modified":"2024-09-19T09:42:05.465135+0000","last_up_change":"2024-09-19T09:29:52.777930+0000","last_in_change":"2024-09-19T09:29:29.573701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:26:05.815869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:30:39.464809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:30:41.429176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8eb04994-c2e4-4428-8ec0-0a9597ab9498","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6803","nonce":1911211878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6805","nonce":1911211878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6809","nonce":1911211878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":1911211878},{"type":"v1","addr":"172.21.15.28:6807","nonce":1911211878}]},"public_addr":"172.21.15.28:6803/1911211878","cluster_addr":"172.21.15.28:6805/1911211878","heartbeat_back_addr":"172.21.15.28:6809/1911211878","heartbeat_front_addr":"172.21.15.28:6807/1911211878","state":["exists","up"]},{"osd":1,"uuid":"c1e8a2f4-f1cf-48d0-af7d-f0e552676eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6801","nonce":3199007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6803","nonce":3199007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6807","nonce":3199007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3199007949},{"type":"v1","addr":"172.21.15.38:6805","nonce":3199007949}]},"public_addr":"172.21.15.38:6801/3199007949","cluster_addr":"172.21.15.38:6803/3199007949","heartbeat_back_addr":"172.21.15.38:6807/3199007949","heartbeat_front_addr":"172.21.15.38:6805/3199007949","state":["exists","up"]},{"osd":2,"uuid":"ae120b16-9641-4ffb-96b8-14da465baef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6801","nonce":720593762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6803","nonce":720593762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6807","nonce":720593762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":720593762},{"type":"v1","addr":"172.21.15.78:6805","nonce":720593762}]},"public_addr":"172.21.15.78:6801/720593762","cluster_addr":"172.21.15.78:6803/720593762","heartbeat_back_addr":"172.21.15.78:6807/720593762","heartbeat_front_addr":"172.21.15.78:6805/720593762","state":["exists","up"]},{"osd":3,"uuid":"225d1570-ea3d-414c-93b3-f440f93bf3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6811","nonce":3620512995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6813","nonce":3620512995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6817","nonce":3620512995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3620512995},{"type":"v1","addr":"172.21.15.28:6815","nonce":3620512995}]},"public_addr":"172.21.15.28:6811/3620512995","cluster_addr":"172.21.15.28:6813/3620512995","heartbeat_back_addr":"172.21.15.28:6817/3620512995","heartbeat_front_addr":"172.21.15.28:6815/3620512995","state":["exists","up"]},{"osd":4,"uuid":"7c9ef00f-d335-4c90-8ad3-541a898ddb60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6809","nonce":465897146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6811","nonce":465897146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6815","nonce":465897146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":465897146},{"type":"v1","addr":"172.21.15.38:6813","nonce":465897146}]},"public_addr":"172.21.15.38:6809/465897146","cluster_addr":"172.21.15.38:6811/465897146","heartbeat_back_addr":"172.21.15.38:6815/465897146","heartbeat_front_addr":"172.21.15.38:6813/465897146","state":["exists","up"]},{"osd":5,"uuid":"05e37b87-069a-444c-9428-a4f59d08116d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6809","nonce":3861457216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6811","nonce":3861457216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6815","nonce":3861457216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3861457216},{"type":"v1","addr":"172.21.15.78:6813","nonce":3861457216}]},"public_addr":"172.21.15.78:6809/3861457216","cluster_addr":"172.21.15.78:6811/3861457216","heartbeat_back_addr":"172.21.15.78:6815/3861457216","heartbeat_front_addr":"172.21.15.78:6813/3861457216","state":["exists","up"]},{"osd":6,"uuid":"d3ab4701-d10e-41f9-a1fb-e9f7e35e52b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6819","nonce":2855006041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6821","nonce":2855006041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6825","nonce":2855006041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2855006041},{"type":"v1","addr":"172.21.15.28:6823","nonce":2855006041}]},"public_addr":"172.21.15.28:6819/2855006041","cluster_addr":"172.21.15.28:6821/2855006041","heartbeat_back_addr":"172.21.15.28:6825/2855006041","heartbeat_front_addr":"172.21.15.28:6823/2855006041","state":["exists","up"]},{"osd":7,"uuid":"e19962bf-fa07-4593-96c4-8f8572d156dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6817","nonce":917618380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6819","nonce":917618380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6823","nonce":917618380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":917618380},{"type":"v1","addr":"172.21.15.38:6821","nonce":917618380}]},"public_addr":"172.21.15.38:6817/917618380","cluster_addr":"172.21.15.38:6819/917618380","heartbeat_back_addr":"172.21.15.38:6823/917618380","heartbeat_front_addr":"172.21.15.38:6821/917618380","state":["exists","up"]},{"osd":8,"uuid":"313b7ee6-931b-4eae-afa0-1f0950c1a0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6817","nonce":1616708891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6819","nonce":1616708891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6823","nonce":1616708891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1616708891},{"type":"v1","addr":"172.21.15.78:6821","nonce":1616708891}]},"public_addr":"172.21.15.78:6817/1616708891","cluster_addr":"172.21.15.78:6819/1616708891","heartbeat_back_addr":"172.21.15.78:6823/1616708891","heartbeat_front_addr":"172.21.15.78:6821/1616708891","state":["exists","up"]},{"osd":9,"uuid":"a6ecdd64-1d04-4526-bf82-921f29aecf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6827","nonce":1812921948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6829","nonce":1812921948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6833","nonce":1812921948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1812921948},{"type":"v1","addr":"172.21.15.28:6831","nonce":1812921948}]},"public_addr":"172.21.15.28:6827/1812921948","cluster_addr":"172.21.15.28:6829/1812921948","heartbeat_back_addr":"172.21.15.28:6833/1812921948","heartbeat_front_addr":"172.21.15.28:6831/1812921948","state":["exists","up"]},{"osd":10,"uuid":"20cc99ff-e67d-4b7a-b4da-e54e634584e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6825","nonce":1524737905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6827","nonce":1524737905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6831","nonce":1524737905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1524737905},{"type":"v1","addr":"172.21.15.38:6829","nonce":1524737905}]},"public_addr":"172.21.15.38:6825/1524737905","cluster_addr":"172.21.15.38:6827/1524737905","heartbeat_back_addr":"172.21.15.38:6831/1524737905","heartbeat_front_addr":"172.21.15.38:6829/1524737905","state":["exists","up"]},{"osd":11,"uuid":"3515d175-e41e-41d0-80d3-bc30d0657a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6825","nonce":2321429454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6827","nonce":2321429454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6831","nonce":2321429454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2321429454},{"type":"v1","addr":"172.21.15.78:6829","nonce":2321429454}]},"public_addr":"172.21.15.78:6825/2321429454","cluster_addr":"172.21.15.78:6827/2321429454","heartbeat_back_addr":"172.21.15.78:6831/2321429454","heartbeat_front_addr":"172.21.15.78:6829/2321429454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:10.736364+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:25:36.498305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:02.434386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:27.763379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:26:54.395630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:19.485924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:27:44.998928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:10.469682+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:28:35.021162+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:00.515350+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:26.654281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:29:51.722710+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:0/4138562507":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/78423353":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/1486296346":"2024-09-20T09:23:47.303309+0000","172.21.15.28:6801/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/1042365655":"2024-09-20T09:23:22.243952+0000","172.21.15.28:6800/3407590911":"2024-09-20T09:23:47.303309+0000","172.21.15.28:0/2768039084":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/3190205246":"2024-09-20T09:23:22.243952+0000","172.21.15.28:0/2766061860":"2024-09-20T09:23:22.243952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:42:41.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 msgr2=0x7f5c280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:41.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 0x7f5c280a5700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005fd0 tx=0x7f5c2c005850 comp rx=0 tx=0).stop 2024-09-19T09:42:41.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 msgr2=0x7f5c3c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9f0 tx=0x7f5c3800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 shutdown_connections 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5c280a3240 0x7f5c280a5700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c3c101420 0x7f5c3c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c3c106a80 0x7f5c3c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 --2- 172.21.15.28:0/4165317479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c3c100a80 0x7f5c3c198420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:41.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 >> 172.21.15.28:0/4165317479 conn(0x7f5c3c0fc7a0 msgr2=0x7f5c3c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:41.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 shutdown_connections 2024-09-19T09:42:41.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:41.438+0000 7f5c447f4640 1 -- 172.21.15.28:0/4165317479 wait complete. 2024-09-19T09:42:41.514 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:41 smithi028 ceph-mon[30972]: pgmap v641: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:42:41.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:41 smithi078 ceph-mon[35619]: pgmap v641: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:42:41.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:41 smithi038 ceph-mon[35271]: pgmap v641: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:42:42.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T09:42:42.314 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:42.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:42 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/4165317479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:42.691 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:42 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/4165317479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:42.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 -- 172.21.15.28:0/4165743642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 msgr2=0x7f5cdc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/4165743642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc106e60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009930 tx=0x7f5ccc02a420 comp rx=0 tx=0).stop 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 -- 172.21.15.28:0/4165743642 shutdown_connections 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/4165743642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc101420 0x7f5cdc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/4165743642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc100a80 0x7f5cdc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/4165743642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc106e60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 -- 172.21.15.28:0/4165743642 >> 172.21.15.28:0/4165743642 conn(0x7f5cdc0fc7a0 msgr2=0x7f5cdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 -- 172.21.15.28:0/4165743642 shutdown_connections 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.774+0000 7f5ce0e55640 1 -- 172.21.15.28:0/4165743642 wait complete. 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 Processor -- start 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 -- start start 2024-09-19T09:42:42.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc100a80 0x7f5cdc0fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc101420 0x7f5cdc0fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc110ec0 con 0x7f5cdc106a80 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc110d40 con 0x7f5cdc100a80 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5ce0e55640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc111040 con 0x7f5cdc101420 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.775+0000 7f5cdad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33488/0 (socket says 172.21.15.28:33488) 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 -- 172.21.15.28:0/1611640468 learned_addr learned my addr 172.21.15.28:0/1611640468 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cda59c640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc100a80 0x7f5cdc0fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cd9d9b640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc101420 0x7f5cdc0fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 -- 172.21.15.28:0/1611640468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc101420 msgr2=0x7f5cdc0fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc101420 0x7f5cdc0fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:42.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 -- 172.21.15.28:0/1611640468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc100a80 msgr2=0x7f5cdc0fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc100a80 0x7f5cdc0fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 -- 172.21.15.28:0/1611640468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc1a6a00 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5cdad9d640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400cad0 tx=0x7f5cd400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400ad60 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc1a6cf0 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd4005020 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400f6b0 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.776+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc1a7230 con 0x7f5cdc106a80 2024-09-19T09:42:42.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.777+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cdc100ee0 con 0x7f5cdc106a80 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.779+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5cd4005560 con 0x7f5cdc106a80 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.779+0000 7f5ccb7fe640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 0x7f5cc40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.780+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f5cd40cfbc0 con 0x7f5cdc106a80 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.780+0000 7f5cda59c640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 0x7f5cc40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.780+0000 7f5cda59c640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 0x7f5cc40a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009900 tx=0x7f5ccc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:42.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.783+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cd4014030 con 0x7f5cdc106a80 2024-09-19T09:42:42.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:42 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/4165317479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:42:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.995+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5cdc0fe360 con 0x7f5cdc106a80 2024-09-19T09:42:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:42.997+0000 7f5ccb7fe640 1 -- 172.21.15.28:0/1611640468 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7f5cdc0fe360 con 0x7f5cdc106a80 2024-09-19T09:42:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T09:42:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 msgr2=0x7f5cc40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 0x7f5cc40a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009900 tx=0x7f5ccc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 msgr2=0x7f5cdc0fd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400cad0 tx=0x7f5cd400ced0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 shutdown_connections 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f5cc40a3240 0x7f5cc40a5700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cdc106a80 0x7f5cdc0fd8b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc101420 0x7f5cdc0fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 --2- 172.21.15.28:0/1611640468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cdc100a80 0x7f5cdc0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.001+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 >> 172.21.15.28:0/1611640468 conn(0x7f5cdc0fc7a0 msgr2=0x7f5cdc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.002+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 shutdown_connections 2024-09-19T09:42:43.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:43.002+0000 7f5ce0e55640 1 -- 172.21.15.28:0/1611640468 wait complete. 2024-09-19T09:42:43.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:42:42.995+0000 7f188457a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:42:43.265 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.265 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.265 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mds[71606]: -- 172.21.15.28:0/736014127 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557bdb67860 con 0x5557bdb45400 2024-09-19T09:42:43.265 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: starting mds.a at 2024-09-19T09:42:43.266 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.266 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.266 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mds[71187]: -- 172.21.15.28:0/238030220 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8708f5860 con 0x55c8708d3000 2024-09-19T09:42:43.266 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: starting mds.g at 2024-09-19T09:42:43.357 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mds[66067]: -- 172.21.15.78:0/1452387604 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55679cec9860 con 0x55679cea7400 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: starting mds.f at 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mds[66667]: -- 172.21.15.78:0/604983637 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4b5843860 con 0x55a4b5821000 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: starting mds.i at 2024-09-19T09:42:43.358 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mds[66369]: -- 172.21.15.78:0/388143519 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649b98bd860 con 0x5649b989b000 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: starting mds.c at 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mds[70022]: -- 172.21.15.78:0/3378786895 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bce93bc20 con 0x563bce918800 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: starting mds.l at 2024-09-19T09:42:43.359 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: 2024-09-19T09:42:43.082+0000 7f3ddeab1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-09-19T09:42:43.369 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.369 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.369 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mds[64982]: -- 172.21.15.38:0/989541377 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2516f1860 con 0x55b2516cf400 2024-09-19T09:42:43.369 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: starting mds.k at 2024-09-19T09:42:43.369 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mds[65294]: -- 172.21.15.38:0/1613229376 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56545a5cb860 con 0x56545a5a9000 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: starting mds.e at 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mds[65915]: -- 172.21.15.38:0/190923002 <== mon.2 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559846fb3c20 con 0x559846f91400 2024-09-19T09:42:43.370 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: starting mds.b at 2024-09-19T09:42:43.371 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: ignoring --setuser ceph since I am not root 2024-09-19T09:42:43.371 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: ignoring --setgroup ceph since I am not root 2024-09-19T09:42:43.371 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mds[65598]: -- 172.21.15.38:0/52401203 <== mon.0 v2:172.21.15.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd029adc20 con 0x55bd0298a800 2024-09-19T09:42:43.371 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: starting mds.h at 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: pgmap v642: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:42:43.627 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:43 smithi028 ceph-mon[30972]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:42:43.629 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: pgmap v642: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:42:43.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:43 smithi038 ceph-mon[35271]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:42:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: pgmap v642: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:42:43.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/1611640468' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:42:43.858 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:43 smithi078 ceph-mon[35619]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:42:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 -- 172.21.15.28:0/3130049977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1784102d40 msgr2=0x7f1784103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 --2- 172.21.15.28:0/3130049977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1784102d40 0x7f1784103120 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1774009900 tx=0x7f177402a480 comp rx=0 tx=0).stop 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 -- 172.21.15.28:0/3130049977 shutdown_connections 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 --2- 172.21.15.28:0/3130049977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f178410c170 0x7f178410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 --2- 172.21.15.28:0/3130049977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 0x7f178410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:44.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 --2- 172.21.15.28:0/3130049977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1784102d40 0x7f1784103120 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 -- 172.21.15.28:0/3130049977 >> 172.21.15.28:0/3130049977 conn(0x7f17840fc7a0 msgr2=0x7f17840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 -- 172.21.15.28:0/3130049977 shutdown_connections 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.349+0000 7f1788d49640 1 -- 172.21.15.28:0/3130049977 wait complete. 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 Processor -- start 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 -- start start 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1784102d40 0x7f1784198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 0x7f1784198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1784111f70 con 0x7f178410c170 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1784111df0 con 0x7f1784102d40 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1788d49640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17841120f0 con 0x7f1784103660 2024-09-19T09:42:44.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49284/0 (socket says 172.21.15.28:49284) 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 -- 172.21.15.28:0/623580887 learned_addr learned my addr 172.21.15.28:0/623580887 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1781d9b640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 0x7f1784198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f178259c640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1784102d40 0x7f1784198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 -- 172.21.15.28:0/623580887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 msgr2=0x7f1784198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 0x7f1784198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 -- 172.21.15.28:0/623580887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1784102d40 msgr2=0x7f1784198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1784102d40 0x7f1784198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 -- 172.21.15.28:0/623580887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178419d400 con 0x7f178410c170 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.350+0000 7f1782d9d640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f177c00cce0 tx=0x7f177c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.351+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c00f450 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.351+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177c00fad0 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.351+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178419d6f0 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.351+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c018c10 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.351+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a4fd0 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.352+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1784103120 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.353+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f177c00f5f0 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.354+0000 7f17737fe640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 0x7f176c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.354+0000 7f178259c640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 0x7f176c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.354+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8612+0+0 (secure 0 0 0) 0x7f177c0d0170 con 0x7f178410c170 2024-09-19T09:42:44.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.354+0000 7f178259c640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 0x7f176c0a5700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1774005ab0 tx=0x7f1774005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:44.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.358+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f177c014030 con 0x7f178410c170 2024-09-19T09:42:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:44.569+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1784107240 con 0x7f178410c170 2024-09-19T09:42:45.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: pgmap v644: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6838/638292337,v1:172.21.15.28:6839/638292337] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.28:6840/2754564081,v1:172.21.15.28:6841/2754564081] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6832/3182981366,v1:172.21.15.38:6836/3182981366] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6834/295783987,v1:172.21.15.38:6837/295783987] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6833/2786892251,v1:172.21.15.38:6835/2786892251] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.38:6838/3023665521,v1:172.21.15.38:6839/3023665521] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6832/1366624727,v1:172.21.15.78:6834/1366624727] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6833/1952148134,v1:172.21.15.78:6835/1952148134] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6837/194886438,v1:172.21.15.78:6838/194886438] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: mds.? [v2:172.21.15.78:6836/1919888656,v1:172.21.15.78:6839/1919888656] up:boot 2024-09-19T09:42:45.265 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:42:45.266 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:45 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:42:45.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: pgmap v644: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6838/638292337,v1:172.21.15.28:6839/638292337] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.28:6840/2754564081,v1:172.21.15.28:6841/2754564081] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6832/3182981366,v1:172.21.15.38:6836/3182981366] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6834/295783987,v1:172.21.15.38:6837/295783987] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6833/2786892251,v1:172.21.15.38:6835/2786892251] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.38:6838/3023665521,v1:172.21.15.38:6839/3023665521] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6832/1366624727,v1:172.21.15.78:6834/1366624727] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6833/1952148134,v1:172.21.15.78:6835/1952148134] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6837/194886438,v1:172.21.15.78:6838/194886438] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: mds.? [v2:172.21.15.78:6836/1919888656,v1:172.21.15.78:6839/1919888656] up:boot 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:42:45.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:42:45.359 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:45 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:42:45.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.369+0000 7f17737fe640 1 -- 172.21.15.28:0/623580887 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f177c09ccb0 con 0x7f178410c170 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 msgr2=0x7f176c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 0x7f176c0a5700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1774005ab0 tx=0x7f1774005a20 comp rx=0 tx=0).stop 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 msgr2=0x7f178419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f177c00cce0 tx=0x7f177c00a620 comp rx=0 tx=0).stop 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 shutdown_connections 2024-09-19T09:42:45.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f176c0a3240 0x7f176c0a5700 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f178410c170 0x7f178419cd20 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1784103660 0x7f1784198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 --2- 172.21.15.28:0/623580887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1784102d40 0x7f1784198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 >> 172.21.15.28:0/623580887 conn(0x7f17840fc7a0 msgr2=0x7f17840fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 shutdown_connections 2024-09-19T09:42:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:45.372+0000 7f1788d49640 1 -- 172.21.15.28:0/623580887 wait complete. 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: pgmap v644: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6838/638292337,v1:172.21.15.28:6839/638292337] up:boot 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.28:6840/2754564081,v1:172.21.15.28:6841/2754564081] up:boot 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6832/3182981366,v1:172.21.15.38:6836/3182981366] up:boot 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6834/295783987,v1:172.21.15.38:6837/295783987] up:boot 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6833/2786892251,v1:172.21.15.38:6835/2786892251] up:boot 2024-09-19T09:42:45.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.38:6838/3023665521,v1:172.21.15.38:6839/3023665521] up:boot 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6832/1366624727,v1:172.21.15.78:6834/1366624727] up:boot 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6833/1952148134,v1:172.21.15.78:6835/1952148134] up:boot 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6837/194886438,v1:172.21.15.78:6838/194886438] up:boot 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: mds.? [v2:172.21.15.78:6836/1919888656,v1:172.21.15.78:6839/1919888656] up:boot 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:42:45.427 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:42:45.428 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:42:45.428 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:42:45.428 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:42:45.428 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='mgr.14150 172.21.15.28:0/3988119089' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:42:45.428 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:45 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:42:46.025 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T09:42:46.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:46 smithi028 ceph-mon[30972]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:42:46.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:46 smithi028 ceph-mon[30972]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:42:46.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:46 smithi028 ceph-mon[30972]: Cluster is now healthy 2024-09-19T09:42:46.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:46 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:42:46.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:46 smithi028 ceph-mon[30972]: fsmap 12 up:standby 2024-09-19T09:42:46.289 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:46.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:46 smithi078 ceph-mon[35619]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:42:46.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:46 smithi078 ceph-mon[35619]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:42:46.357 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:46 smithi078 ceph-mon[35619]: Cluster is now healthy 2024-09-19T09:42:46.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:46 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:42:46.358 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:46 smithi078 ceph-mon[35619]: fsmap 12 up:standby 2024-09-19T09:42:46.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:46 smithi038 ceph-mon[35271]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:42:46.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:46 smithi038 ceph-mon[35271]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:42:46.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:46 smithi038 ceph-mon[35271]: Cluster is now healthy 2024-09-19T09:42:46.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:46 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/623580887' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:42:46.426 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:46 smithi038 ceph-mon[35271]: fsmap 12 up:standby 2024-09-19T09:42:46.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 -- 172.21.15.28:0/3460255780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 msgr2=0x7f67bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:46.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/3460255780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc10d950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009cd0 tx=0x7f67b402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 -- 172.21.15.28:0/3460255780 shutdown_connections 2024-09-19T09:42:46.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/3460255780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc100a80 0x7f67bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/3460255780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc10d950 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/3460255780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc106a80 0x7f67bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 -- 172.21.15.28:0/3460255780 >> 172.21.15.28:0/3460255780 conn(0x7f67bc0fc7a0 msgr2=0x7f67bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 -- 172.21.15.28:0/3460255780 shutdown_connections 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.716+0000 7f67c1dd1640 1 -- 172.21.15.28:0/3460255780 wait complete. 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.717+0000 7f67c1dd1640 1 Processor -- start 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.717+0000 7f67c1dd1640 1 -- start start 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.717+0000 7f67c1dd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc100a80 0x7f67bc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.717+0000 7f67c1dd1640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.717+0000 7f67c1dd1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc106a80 0x7f67bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67c1dd1640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc110e80 con 0x7f67bc101420 2024-09-19T09:42:46.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67c1dd1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc110d00 con 0x7f67bc106a80 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67c1dd1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc111000 con 0x7f67bc100a80 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49314/0 (socket says 172.21.15.28:49314) 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 -- 172.21.15.28:0/2700052776 learned_addr learned my addr 172.21.15.28:0/2700052776 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67bb7fe640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc100a80 0x7f67bc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67bbfff640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc106a80 0x7f67bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 -- 172.21.15.28:0/2700052776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc100a80 msgr2=0x7f67bc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc100a80 0x7f67bc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 -- 172.21.15.28:0/2700052776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc106a80 msgr2=0x7f67bc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc106a80 0x7f67bc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 -- 172.21.15.28:0/2700052776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc19d440 con 0x7f67bc101420 2024-09-19T09:42:46.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b2ffd640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f67a400d950 tx=0x7f67a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:46.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4017030 con 0x7f67bc101420 2024-09-19T09:42:46.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc19d730 con 0x7f67bc101420 2024-09-19T09:42:46.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a4004020 con 0x7f67bc101420 2024-09-19T09:42:46.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.718+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40047a0 con 0x7f67bc101420 2024-09-19T09:42:46.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.719+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a5010 con 0x7f67bc101420 2024-09-19T09:42:46.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.719+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc100ee0 con 0x7f67bc101420 2024-09-19T09:42:46.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.721+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f67a40041d0 con 0x7f67bc101420 2024-09-19T09:42:46.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.722+0000 7f67b97fa640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:46.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.722+0000 7f67bb7fe640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:46.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.722+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8612+0+0 (secure 0 0 0) 0x7f67a40cfb10 con 0x7f67bc101420 2024-09-19T09:42:46.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.722+0000 7f67bb7fe640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 0x7f67a80a5700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005fd0 tx=0x7f67ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:46.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.725+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f67a409c6b0 con 0x7f67bc101420 2024-09-19T09:42:46.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:46.905+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f67bc10a060 con 0x7f67bc101420 2024-09-19T09:42:47.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.379+0000 7f67b97fa640 1 -- 172.21.15.28:0/2700052776 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7f67a409bfb0 con 0x7f67bc101420 2024-09-19T09:42:47.381 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 msgr2=0x7f67a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 0x7f67a80a5700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005fd0 tx=0x7f67ac00b390 comp rx=0 tx=0).stop 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 msgr2=0x7f67bc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f67a400d950 tx=0x7f67a400de10 comp rx=0 tx=0).stop 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 shutdown_connections 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67bc106a80 0x7f67bc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:47.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67bc101420 0x7f67bc1989d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:47.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 --2- 172.21.15.28:0/2700052776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc100a80 0x7f67bc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:47.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 >> 172.21.15.28:0/2700052776 conn(0x7f67bc0fc7a0 msgr2=0x7f67bc06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:47.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.382+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 shutdown_connections 2024-09-19T09:42:47.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:47.383+0000 7f67c1dd1640 1 -- 172.21.15.28:0/2700052776 wait complete. 2024-09-19T09:42:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:47 smithi038 ceph-mon[35271]: pgmap v645: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:42:47.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:47 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:47.703 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:47 smithi028 ceph-mon[30972]: pgmap v645: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:42:47.703 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:47 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:47.729 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:47 smithi078 ceph-mon[35619]: pgmap v645: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-19T09:42:47.729 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:47 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:47.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T09:42:48.236 INFO:teuthology.orchestra.run.smithi028.stderr:Inferring config /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:42:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:48 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:48.607 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:48 smithi078 ceph-mon[35619]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:42:48.640 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:48 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:48.640 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:48 smithi028 ceph-mon[30972]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.650+0000 7f3341b95640 1 -- 172.21.15.28:0/1408410029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c106a80 msgr2=0x7f333c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.650+0000 7f3341b95640 1 --2- 172.21.15.28:0/1408410029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c106a80 0x7f333c106e60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f332c009930 tx=0x7f332c02a420 comp rx=0 tx=0).stop 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 -- 172.21.15.28:0/1408410029 shutdown_connections 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 --2- 172.21.15.28:0/1408410029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c100a80 0x7f333c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 --2- 172.21.15.28:0/1408410029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c101420 0x7f333c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 --2- 172.21.15.28:0/1408410029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c106a80 0x7f333c106e60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 -- 172.21.15.28:0/1408410029 >> 172.21.15.28:0/1408410029 conn(0x7f333c0fc7a0 msgr2=0x7f333c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 -- 172.21.15.28:0/1408410029 shutdown_connections 2024-09-19T09:42:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 -- 172.21.15.28:0/1408410029 wait complete. 2024-09-19T09:42:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.651+0000 7f3341b95640 1 Processor -- start 2024-09-19T09:42:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 -- start start 2024-09-19T09:42:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c100a80 0x7f333c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 0x7f333c1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f333c110e80 con 0x7f333c101420 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f333c110d00 con 0x7f333c106a80 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f3341b95640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111000 con 0x7f333c100a80 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49334/0 (socket says 172.21.15.28:49334) 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 -- 172.21.15.28:0/2075007719 learned_addr learned my addr 172.21.15.28:0/2075007719 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333b7fe640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c100a80 0x7f333c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333bfff640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 0x7f333c1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 -- 172.21.15.28:0/2075007719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c100a80 msgr2=0x7f333c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c100a80 0x7f333c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 -- 172.21.15.28:0/2075007719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 msgr2=0x7f333c1a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 0x7f333c1a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333affd640 1 -- 172.21.15.28:0/2075007719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c1a4830 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.652+0000 7f333bfff640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 0x7f333c1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f333affd640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f333000e9f0 tx=0x7f333000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333000cd80 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3330010910 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1a4a00 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330018990 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.653+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a4f40 con 0x7f333c101420 2024-09-19T09:42:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.654+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c100ee0 con 0x7f333c101420 2024-09-19T09:42:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.655+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3330010ac0 con 0x7f333c101420 2024-09-19T09:42:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.656+0000 7f3338ff9640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 0x7f33240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:42:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.656+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8087+0+0 (secure 0 0 0) 0x7f33300cfb40 con 0x7f333c101420 2024-09-19T09:42:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.656+0000 7f333b7fe640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 0x7f33240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:42:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.656+0000 7f333b7fe640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 0x7f33240a5780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f332c009900 tx=0x7f332c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:42:48.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.661+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f333009c940 con 0x7f333c101420 2024-09-19T09:42:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:48 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2700052776' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:48.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:48 smithi038 ceph-mon[35271]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:42:48.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:48.837+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f333c108ff0 con 0x7f333c101420 2024-09-19T09:42:49.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.388+0000 7f3338ff9640 1 -- 172.21.15.28:0/2075007719 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7f333009c240 con 0x7f333c101420 2024-09-19T09:42:49.390 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-19T09:42:49.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.393+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 msgr2=0x7f33240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:49.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.393+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 0x7f33240a5780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f332c009900 tx=0x7f332c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.393+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 msgr2=0x7f333c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:42:49.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.393+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f333000e9f0 tx=0x7f333000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.393+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 shutdown_connections 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:6800/147346946,v1:172.21.15.28:6801/147346946] conn(0x7f33240a32c0 0x7f33240a5780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f333c106a80 0x7f333c1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333c101420 0x7f333c1997b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 --2- 172.21.15.28:0/2075007719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f333c100a80 0x7f333c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 >> 172.21.15.28:0/2075007719 conn(0x7f333c0fc7a0 msgr2=0x7f333c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 shutdown_connections 2024-09-19T09:42:49.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T09:42:49.394+0000 7f3341b95640 1 -- 172.21.15.28:0/2075007719 wait complete. 2024-09-19T09:42:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:49 smithi038 ceph-mon[35271]: pgmap v647: 65 pgs: 65 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:42:49.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:49 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:49.727 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:49 smithi028 ceph-mon[30972]: pgmap v647: 65 pgs: 65 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:42:49.728 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:49 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:49 smithi078 ceph-mon[35619]: pgmap v647: 65 pgs: 65 active+clean; 577 KiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:42:49.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:49 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:42:50.048 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:42:50.076 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:42:50.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:42:50.171 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T09:42:50.171 DEBUG:teuthology.orchestra.run.smithi028:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:42:50.187 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:42:50.210 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:42:50.226 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T09:42:50.227 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T09:42:50.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a 2024-09-19T09:42:50.549 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 systemd[1]: Stopping Ceph mon.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:50.550 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 ceph-mon[30972]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:50.550 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 ceph-mon[30972]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:42:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:50 smithi038 ceph-mon[35271]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:50.676 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:50 smithi038 ceph-mon[35271]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:42:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:50 smithi078 ceph-mon[35619]: from='client.? 172.21.15.28:0/2075007719' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:42:50.857 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:50 smithi078 ceph-mon[35619]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:42:50.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:42:50.547+0000 7f1889d85640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:42:50.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a[30968]: 2024-09-19T09:42:50.547+0000 7f1889d85640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T09:42:50.933 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:50 smithi028 podman[102862]: 2024-09-19 09:42:50.650003198 +0000 UTC m=+0.226811693 container died 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T09:42:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:51 smithi028 podman[102862]: 2024-09-19 09:42:51.131090078 +0000 UTC m=+0.707898562 container cleanup 5e0e81b5a83feb181ab69c99b7b1822e0a2e31c12fee524b91ecb205417cb81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T09:42:51.264 INFO:journalctl@ceph.mon.a.smithi028.stdout:Sep 19 09:42:51 smithi028 bash[102862]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-a 2024-09-19T09:42:51.593 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T09:42:51.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:51.642 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T09:42:51.642 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T09:42:51.642 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.b 2024-09-19T09:42:51.926 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:51 smithi038 systemd[1]: Stopping Ceph mon.b for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:52.249 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-b[35267]: 2024-09-19T09:42:51.946+0000 7fe99fb09640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:42:52.249 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-b[35267]: 2024-09-19T09:42:51.946+0000 7fe99fb09640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:42:52.249 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:51 smithi038 podman[69924]: 2024-09-19 09:42:51.988089448 +0000 UTC m=+0.145368819 container died 9fa06e52672b41efc7bb05b486efabbfe1e891ff1a0158e787c0bd561b133948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T09:42:52.615 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:52 smithi038 podman[69924]: 2024-09-19 09:42:52.338334811 +0000 UTC m=+0.495614163 container cleanup 9fa06e52672b41efc7bb05b486efabbfe1e891ff1a0158e787c0bd561b133948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:42:52.616 INFO:journalctl@ceph.mon.b.smithi038.stdout:Sep 19 09:42:52 smithi038 bash[69924]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-b 2024-09-19T09:42:52.751 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T09:42:52.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:52.797 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T09:42:52.797 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T09:42:52.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.c 2024-09-19T09:42:53.107 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:52 smithi078 systemd[1]: Stopping Ceph mon.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:53.568 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c[35615]: 2024-09-19T09:42:53.106+0000 7f7cc636a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:42:53.568 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c[35615]: 2024-09-19T09:42:53.106+0000 7f7cc636a640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:42:53.568 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 podman[71207]: 2024-09-19 09:42:53.331739273 +0000 UTC m=+0.328380488 container died 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:42:53.860 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 podman[71207]: 2024-09-19 09:42:53.66505303 +0000 UTC m=+0.661694247 container cleanup 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T09:42:53.860 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 bash[71207]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c 2024-09-19T09:42:54.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T09:42:54.169 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:53 smithi078 podman[71219]: 2024-09-19 09:42:53.916410305 +0000 UTC m=+0.582837613 container remove 819d566cd6fea877c5ac7a3ef78569d8e6e3ae35aa72fe478ddfeed833cda392 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T09:42:54.169 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:54 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-19T09:42:54.169 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:54 smithi078 systemd[1]: Stopped Ceph mon.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:54.169 INFO:journalctl@ceph.mon.c.smithi078.stdout:Sep 19 09:42:54 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mon.c.service: Consumed 11.561s CPU time. 2024-09-19T09:42:54.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:54.189 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T09:42:54.189 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T09:42:54.189 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x 2024-09-19T09:42:54.514 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:54 smithi028 systemd[1]: Stopping Ceph mgr.x for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:54.825 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:54 smithi028 podman[103334]: 2024-09-19 09:42:54.553816573 +0000 UTC m=+0.150147037 container died 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T09:42:55.184 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:54 smithi028 podman[103334]: 2024-09-19 09:42:54.981475625 +0000 UTC m=+0.577806119 container cleanup 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T09:42:55.184 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:54 smithi028 bash[103334]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x 2024-09-19T09:42:55.419 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T09:42:55.443 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:55 smithi028 podman[103346]: 2024-09-19 09:42:55.183046264 +0000 UTC m=+0.625792257 container remove 598dadb37a24127268f78b13c5d1252fdacb6c4e3215b68ed3d22621a9bc3281 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:42:55.443 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:55 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T09:42:55.443 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:55 smithi028 systemd[1]: Stopped Ceph mgr.x for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:55.443 INFO:journalctl@ceph.mgr.x.smithi028.stdout:Sep 19 09:42:55 smithi028 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.x.service: Consumed 49.117s CPU time. 2024-09-19T09:42:55.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:55.459 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T09:42:55.459 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T09:42:55.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y 2024-09-19T09:42:55.798 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:55 smithi038 systemd[1]: Stopping Ceph mgr.y for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:56.111 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:55 smithi038 podman[70110]: 2024-09-19 09:42:55.797035067 +0000 UTC m=+0.137259009 container died 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:42:56.426 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 podman[70110]: 2024-09-19 09:42:56.201732316 +0000 UTC m=+0.541956299 container cleanup 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T09:42:56.426 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 bash[70110]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y 2024-09-19T09:42:56.787 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T09:42:56.809 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 podman[70123]: 2024-09-19 09:42:56.536121705 +0000 UTC m=+0.735256556 container remove 671ba97058b8cdbd7c59859d80adf16a4b5112e66a80c583ac2c3058c9f6fd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-y, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T09:42:56.809 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-19T09:42:56.809 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-19T09:42:56.809 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 systemd[1]: Stopped Ceph mgr.y for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:56.809 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 19 09:42:56 smithi038 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.y.service: Consumed 6.767s CPU time. 2024-09-19T09:42:56.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:56.822 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T09:42:56.822 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T09:42:56.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z 2024-09-19T09:42:57.107 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:56 smithi078 systemd[1]: Stopping Ceph mgr.z for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:57.456 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:57 smithi078 podman[71330]: 2024-09-19 09:42:57.15710029 +0000 UTC m=+0.142651645 container died d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T09:42:57.839 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:57 smithi078 podman[71330]: 2024-09-19 09:42:57.594963566 +0000 UTC m=+0.580514929 container cleanup d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T09:42:57.839 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:57 smithi078 bash[71330]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z 2024-09-19T09:42:58.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T09:42:58.093 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:57 smithi078 podman[71343]: 2024-09-19 09:42:57.838142401 +0000 UTC m=+0.671906015 container remove d824d11d09e2ac99c76f556ae09a15ffc453282145ac0a32f3a2e93b5da82f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:42:58.094 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:57 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T09:42:58.094 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:58 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T09:42:58.094 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:58 smithi078 systemd[1]: Stopped Ceph mgr.z for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:42:58.094 INFO:journalctl@ceph.mgr.z.smithi078.stdout:Sep 19 09:42:58 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mgr.z.service: Consumed 6.486s CPU time. 2024-09-19T09:42:58.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:42:58.108 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T09:42:58.108 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T09:42:58.108 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.0 2024-09-19T09:42:58.413 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:42:58 smithi028 systemd[1]: Stopping Ceph osd.0 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:42:58.764 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:42:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:42:58.411+0000 7f86639fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:42:58.764 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:42:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:42:58.411+0000 7f86639fe640 -1 osd.0 88 *** Got signal Terminated *** 2024-09-19T09:42:58.764 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:42:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0[44837]: 2024-09-19T09:42:58.411+0000 7f86639fe640 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:04.164 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:03 smithi028 podman[103456]: 2024-09-19 09:43:03.908300181 +0000 UTC m=+5.602407402 container died 9a4fc946c055e8062c89845247a3b31cc4a9c46b04368eb030e86fb64b13033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:43:04.421 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:04 smithi028 podman[103456]: 2024-09-19 09:43:04.252362562 +0000 UTC m=+5.946469781 container cleanup 9a4fc946c055e8062c89845247a3b31cc4a9c46b04368eb030e86fb64b13033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T09:43:04.422 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:04 smithi028 bash[103456]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0 2024-09-19T09:43:04.764 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:04 smithi028 podman[103474]: 2024-09-19 09:43:04.420287268 +0000 UTC m=+0.507809900 container remove 9a4fc946c055e8062c89845247a3b31cc4a9c46b04368eb030e86fb64b13033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T09:43:05.050 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:04 smithi028 podman[103563]: 2024-09-19 09:43:04.734101352 +0000 UTC m=+0.016339241 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:05.050 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:04 smithi028 podman[103563]: 2024-09-19 09:43:04.838127725 +0000 UTC m=+0.120365595 container create 3ffcaf393d6c394d59da5fa470a05d6ca18db62e4e483c0378d311ab9b35240d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T09:43:05.050 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:05 smithi028 podman[103563]: 2024-09-19 09:43:05.048881326 +0000 UTC m=+0.331119196 container init 3ffcaf393d6c394d59da5fa470a05d6ca18db62e4e483c0378d311ab9b35240d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T09:43:05.514 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:05 smithi028 podman[103563]: 2024-09-19 09:43:05.052388983 +0000 UTC m=+0.334626859 container start 3ffcaf393d6c394d59da5fa470a05d6ca18db62e4e483c0378d311ab9b35240d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T09:43:05.515 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:05 smithi028 podman[103563]: 2024-09-19 09:43:05.148849147 +0000 UTC m=+0.431087035 container attach 3ffcaf393d6c394d59da5fa470a05d6ca18db62e4e483c0378d311ab9b35240d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T09:43:05.515 INFO:journalctl@ceph.osd.0.smithi028.stdout:Sep 19 09:43:05 smithi028 podman[103563]: 2024-09-19 09:43:05.2492993 +0000 UTC m=+0.531537180 container died 3ffcaf393d6c394d59da5fa470a05d6ca18db62e4e483c0378d311ab9b35240d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:05.843 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T09:43:05.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:05.886 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T09:43:05.886 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T09:43:05.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.1 2024-09-19T09:43:06.176 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:05 smithi038 systemd[1]: Stopping Ceph osd.1 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:06.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:43:06.338+0000 7fdd12b48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:06.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:43:06.338+0000 7fdd12b48640 -1 osd.1 88 *** Got signal Terminated *** 2024-09-19T09:43:06.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1[39762]: 2024-09-19T09:43:06.338+0000 7fdd12b48640 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:12.176 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:11 smithi038 podman[70235]: 2024-09-19 09:43:11.850220599 +0000 UTC m=+5.762201176 container died 4b8455f6c4fdbafd2c81eb00ecdb6faa21a04d7bbaf21a26c831b52ab7b1e4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T09:43:12.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:12 smithi038 podman[70235]: 2024-09-19 09:43:12.180633746 +0000 UTC m=+6.092614313 container cleanup 4b8455f6c4fdbafd2c81eb00ecdb6faa21a04d7bbaf21a26c831b52ab7b1e4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T09:43:12.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:12 smithi038 bash[70235]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1 2024-09-19T09:43:12.676 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:12 smithi038 podman[70266]: 2024-09-19 09:43:12.374111925 +0000 UTC m=+0.524825742 container remove 4b8455f6c4fdbafd2c81eb00ecdb6faa21a04d7bbaf21a26c831b52ab7b1e4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:13.095 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:12 smithi038 podman[70353]: 2024-09-19 09:43:12.703426086 +0000 UTC m=+0.015944839 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:13.095 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:12 smithi038 podman[70353]: 2024-09-19 09:43:12.808046458 +0000 UTC m=+0.120565197 container create 238d99762b7d1761e0853b08c5252183e92a039283541c817817e36de806111a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:13.095 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:13 smithi038 podman[70353]: 2024-09-19 09:43:13.018690072 +0000 UTC m=+0.331208813 container init 238d99762b7d1761e0853b08c5252183e92a039283541c817817e36de806111a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-19T09:43:13.095 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:13 smithi038 podman[70353]: 2024-09-19 09:43:13.021864859 +0000 UTC m=+0.334383604 container start 238d99762b7d1761e0853b08c5252183e92a039283541c817817e36de806111a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:43:13.095 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:13 smithi038 podman[70353]: 2024-09-19 09:43:13.093624105 +0000 UTC m=+0.406142867 container attach 238d99762b7d1761e0853b08c5252183e92a039283541c817817e36de806111a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T09:43:13.426 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 19 09:43:13 smithi038 podman[70353]: 2024-09-19 09:43:13.212834375 +0000 UTC m=+0.525353127 container died 238d99762b7d1761e0853b08c5252183e92a039283541c817817e36de806111a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-19T09:43:13.829 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T09:43:13.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:13.870 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T09:43:13.870 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T09:43:13.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.2 2024-09-19T09:43:14.183 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:13 smithi078 systemd[1]: Stopping Ceph osd.2 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:14.607 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:43:14.180+0000 7faf35829640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:14.607 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:43:14.180+0000 7faf35829640 -1 osd.2 88 *** Got signal Terminated *** 2024-09-19T09:43:14.607 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2[40475]: 2024-09-19T09:43:14.180+0000 7faf35829640 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:19.836 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:19 smithi078 podman[71454]: 2024-09-19 09:43:19.543701932 +0000 UTC m=+5.480118169 container died 1ccbb63273078a3251950d5050a4805f32d0a716d606291c4f8fe1c45ad0f5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T09:43:20.107 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:19 smithi078 podman[71454]: 2024-09-19 09:43:19.923734276 +0000 UTC m=+5.860150512 container cleanup 1ccbb63273078a3251950d5050a4805f32d0a716d606291c4f8fe1c45ad0f5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:20.107 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:19 smithi078 bash[71454]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2 2024-09-19T09:43:20.520 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71481]: 2024-09-19 09:43:20.208562334 +0000 UTC m=+0.657642890 container remove 1ccbb63273078a3251950d5050a4805f32d0a716d606291c4f8fe1c45ad0f5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:20.847 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71570]: 2024-09-19 09:43:20.42142104 +0000 UTC m=+0.015770115 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:20.847 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71570]: 2024-09-19 09:43:20.534790627 +0000 UTC m=+0.129139696 container create 48c3a09f019060c57f363c061b609465fde530a3e0c08881bc48b06764889795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:20.847 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71570]: 2024-09-19 09:43:20.753566207 +0000 UTC m=+0.347915268 container init 48c3a09f019060c57f363c061b609465fde530a3e0c08881bc48b06764889795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:43:20.847 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71570]: 2024-09-19 09:43:20.756833599 +0000 UTC m=+0.351182660 container start 48c3a09f019060c57f363c061b609465fde530a3e0c08881bc48b06764889795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-19T09:43:20.847 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:20 smithi078 podman[71570]: 2024-09-19 09:43:20.845030282 +0000 UTC m=+0.439379346 container attach 48c3a09f019060c57f363c061b609465fde530a3e0c08881bc48b06764889795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T09:43:21.108 INFO:journalctl@ceph.osd.2.smithi078.stdout:Sep 19 09:43:21 smithi078 podman[71570]: 2024-09-19 09:43:21.0905513 +0000 UTC m=+0.684900362 container died 48c3a09f019060c57f363c061b609465fde530a3e0c08881bc48b06764889795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T09:43:21.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T09:43:21.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:21.772 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T09:43:21.772 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T09:43:21.773 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.3 2024-09-19T09:43:22.082 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:21 smithi028 systemd[1]: Stopping Ceph osd.3 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:22.514 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:22 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:43:22.079+0000 7f666046b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:22.514 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:22 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:43:22.079+0000 7f666046b640 -1 osd.3 88 *** Got signal Terminated *** 2024-09-19T09:43:22.514 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:22 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:43:22.079+0000 7f666046b640 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:24.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:24 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:24.161+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:25.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:25 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:25.151+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:25.846 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:25.555+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:25.846 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:25.470+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:26.014 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:25 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:43:25.588+0000 7f665ca70640 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:02.572747+0000 front 2024-09-19T09:43:02.572772+0000 (oldest deadline 2024-09-19T09:43:24.872967+0000) 2024-09-19T09:43:26.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:26.198+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:26.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:26.581+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:26.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:26.478+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:26.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:26 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:26.477+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:27.014 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:26 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3[51484]: 2024-09-19T09:43:26.594+0000 7f665ca70640 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:02.572747+0000 front 2024-09-19T09:43:02.572772+0000 (oldest deadline 2024-09-19T09:43:24.872967+0000) 2024-09-19T09:43:27.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:27 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:27.204+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:27.824 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:27 smithi028 podman[103676]: 2024-09-19 09:43:27.528821267 +0000 UTC m=+5.553835968 container stop 55f2fe4a0e53799c8c4ad51f6aa6e6884ae3e847f60e877be2b0f7036974a6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T09:43:27.824 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:27 smithi028 podman[103676]: 2024-09-19 09:43:27.554830547 +0000 UTC m=+5.579845250 container died 55f2fe4a0e53799c8c4ad51f6aa6e6884ae3e847f60e877be2b0f7036974a6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:27.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:27.588+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:27.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:27.463+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:27.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:27 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:27.460+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:28.098 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:27 smithi028 podman[103676]: 2024-09-19 09:43:27.921209908 +0000 UTC m=+5.946224607 container cleanup 55f2fe4a0e53799c8c4ad51f6aa6e6884ae3e847f60e877be2b0f7036974a6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T09:43:28.098 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:27 smithi028 bash[103676]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3 2024-09-19T09:43:28.514 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103690]: 2024-09-19 09:43:28.097421447 +0000 UTC m=+0.561397788 container remove 55f2fe4a0e53799c8c4ad51f6aa6e6884ae3e847f60e877be2b0f7036974a6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:28.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:28.227+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:28.832 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.441977044 +0000 UTC m=+0.016443063 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:28.832 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.556661011 +0000 UTC m=+0.131127028 container create 90d8d615853af8d02f113fb0c8c34794201539c3de8300ff3881c744d1f707e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T09:43:28.833 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.750663151 +0000 UTC m=+0.325129169 container init 90d8d615853af8d02f113fb0c8c34794201539c3de8300ff3881c744d1f707e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:28.833 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.754132449 +0000 UTC m=+0.328598468 container start 90d8d615853af8d02f113fb0c8c34794201539c3de8300ff3881c744d1f707e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:43:28.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:28.436+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:28.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:28.600+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:28.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:28 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:28.427+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:29.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:28 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:28.831+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:29.264 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.83375835 +0000 UTC m=+0.408224384 container attach 90d8d615853af8d02f113fb0c8c34794201539c3de8300ff3881c744d1f707e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T09:43:29.264 INFO:journalctl@ceph.osd.3.smithi028.stdout:Sep 19 09:43:28 smithi028 podman[103782]: 2024-09-19 09:43:28.953642696 +0000 UTC m=+0.528108725 container died 90d8d615853af8d02f113fb0c8c34794201539c3de8300ff3881c744d1f707e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T09:43:29.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:29 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:29.262+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:29.595 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T09:43:29.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:29.637 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T09:43:29.637 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T09:43:29.637 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.4 2024-09-19T09:43:29.685 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:29 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:29.424+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:29.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:29 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:29.630+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:29.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:29 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:29.440+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:30.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:29 smithi038 systemd[1]: Stopping Ceph osd.4 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:30.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:29 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:43:29.932+0000 7fb4a0e1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:30.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:29 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:43:29.932+0000 7fb4a0e1d640 -1 osd.4 88 *** Got signal Terminated *** 2024-09-19T09:43:30.176 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:29 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4[46639]: 2024-09-19T09:43:29.932+0000 7fb4a0e1d640 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:30.218 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:29 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:29.867+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:30.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:30 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:30.216+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:30.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:30 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:30.473+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:30.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:30 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:30.625+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:30.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:30 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:30.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:31.200 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:30 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:30.899+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:31.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:31 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:31.199+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:31.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:31 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:31.410+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:31.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:31 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:31.612+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:31.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:31 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:31.487+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:32.242 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:31 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:31.934+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:32.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:32 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:32.241+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:32.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:32 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:32.401+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:32.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:32 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:32.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:32.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:32 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:32.587+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:32.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:32 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:32.587+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:33.246 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:32 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:32.905+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:33.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:33 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:33.245+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:33.514 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:33 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:33.245+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:33.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:33 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:33.396+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:33.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:33 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:33.566+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:33.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:33 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:33.566+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:33.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:33 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:33.527+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:33.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:33 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:33.527+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:34.219 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:34 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:34.218+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:34.220 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:33 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:33.875+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:34.507 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:34 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:34.218+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:34.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:34 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:34.371+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:34.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:34.570+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:34.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:34.570+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:34.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:34.483+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:34.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:34.483+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:34.858 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:34 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:34.470+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:35.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:34 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:34.868+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:35.523 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:35 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:35.264+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:35.523 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:35 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:35.264+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:35.619 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:35 smithi038 podman[70458]: 2024-09-19 09:43:35.367235764 +0000 UTC m=+5.533686019 container died 2a81369d1ffc329fc77b7f452a8329ae2d290b360dcf98df05464f4ed21f9833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:35.619 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:35 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:35.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:35.857 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:35 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:35.476+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:35.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:35 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:35.612+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:35.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:35 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:35.612+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:35.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:35 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:35.491+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:35.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:35 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:35.491+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:35.885 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:35 smithi038 podman[70458]: 2024-09-19 09:43:35.699537542 +0000 UTC m=+5.865987780 container cleanup 2a81369d1ffc329fc77b7f452a8329ae2d290b360dcf98df05464f4ed21f9833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:43:35.885 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:35 smithi038 bash[70458]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4 2024-09-19T09:43:36.177 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:35 smithi038 podman[70474]: 2024-09-19 09:43:35.884134386 +0000 UTC m=+0.514416012 container remove 2a81369d1ffc329fc77b7f452a8329ae2d290b360dcf98df05464f4ed21f9833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T09:43:36.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:35 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:35.891+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:36.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:35 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:35.891+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:36.500 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:36 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:36.304+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:36.500 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:36 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:36.304+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:36.500 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.213268007 +0000 UTC m=+0.015694048 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:36.500 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.401668895 +0000 UTC m=+0.204094932 container create fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T09:43:36.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:36.269+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:36.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:36.269+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:36.790 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.620637572 +0000 UTC m=+0.423063614 container init fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T09:43:36.790 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.624154743 +0000 UTC m=+0.426580786 container start fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:36.790 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.69587948 +0000 UTC m=+0.498305528 container attach fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T09:43:36.857 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:36.428+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:36.857 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:36.606+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:36.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:36.606+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:36.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:36.510+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:36.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:36 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:36.510+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:37.141 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:36 smithi038 podman[70565]: 2024-09-19 09:43:36.820147455 +0000 UTC m=+0.622573492 container died fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:37.142 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:36 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:36.788+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:37.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:36.863+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:37.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:36.863+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:37.396 INFO:journalctl@ceph.osd.4.smithi038.stdout:Sep 19 09:43:37 smithi038 podman[70589]: 2024-09-19 09:43:37.374065554 +0000 UTC m=+0.548726965 container remove fc21536d81c18df7cfd4f642a7dde982477e3fa509852701f7e72a3f0e425a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:43:37.396 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:37 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:37.269+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:37.396 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:37 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:37.269+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:37.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T09:43:37.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:37.445 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T09:43:37.445 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T09:43:37.445 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.5 2024-09-19T09:43:37.736 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:37.433+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:37.736 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 systemd[1]: Stopping Ceph osd.5 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:37.736 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:37.589+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:37.589+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:37.734+0000 7f2bf8c4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:37.734+0000 7f2bf8c4b640 -1 osd.5 88 *** Got signal Terminated *** 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:37.734+0000 7f2bf8c4b640 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:37.524+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:37.737 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:37 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:37.524+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:37.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:37 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:37.267+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:37.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:37 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:37.267+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:37.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:37 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:37.806+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:38.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:37 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:37.882+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:38.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:37 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:37.882+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:38.659 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:38 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:38.278+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:38.659 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:38 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:38.278+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:38.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:38.286+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:38.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:38.286+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:38.857 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:38 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:38.398+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:38.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:38 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:38.576+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:38.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:38 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:38.576+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:38.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:38 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:38.572+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:38.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:38 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:38.572+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:38.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:38 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:38.781+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:39.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:38.847+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:39.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:38.847+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:39.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:39 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:39.289+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:39.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:39 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:39.289+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:39.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:39 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:39.271+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:39.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:39 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:39.271+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:39.857 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:39 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:39.391+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:39.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:39 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:39.548+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:39.858 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:39 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:39.548+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:39.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:39 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:39.526+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:39.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:39 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:39.526+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:40.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:39 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:39.735+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:40.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:39 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:39.870+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:40.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:39 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:39.870+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:40.607 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:40 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:40.505+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:40.607 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:40 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:40.505+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:40.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:40 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:40.349+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:40.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:40 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:40.515+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:40.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:40 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:40.515+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:40.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:40 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:40.278+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:40.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:40 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:40.278+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:40.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:40 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:40.295+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:40.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:40 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:40.295+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:41.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:40 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:40.771+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:41.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:40 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:40.908+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:41.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:40 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:40.908+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:41.554 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:41.302+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:41.554 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:41.302+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:41.554 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:41.551+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:41.554 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:41.551+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:41.555 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:41.551+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.737105+0000 front 2024-09-19T09:43:16.737091+0000 (oldest deadline 2024-09-19T09:43:40.837238+0000) 2024-09-19T09:43:41.555 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:41.503+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:41.555 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:41.503+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:41.555 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:41 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:41.503+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:41.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:41 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:41.257+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:41.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:41 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:41.257+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:41.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:41 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:41.257+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:41.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:41 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:41.305+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:41.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:41 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:41.305+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:42.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:41 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:41.762+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:42.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:41 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:41.762+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:42.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:41 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:41.898+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:42.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:41 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:41.898+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:42.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:41 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:41.898+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:42.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:42.322+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:42.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:42.322+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:42.530+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.634405+0000 front 2024-09-19T09:43:01.634320+0000 (oldest deadline 2024-09-19T09:43:25.134386+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:42.530+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.735979+0000 front 2024-09-19T09:43:09.736031+0000 (oldest deadline 2024-09-19T09:43:32.036047+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5[47456]: 2024-09-19T09:43:42.530+0000 7f2bf5250640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.737105+0000 front 2024-09-19T09:43:16.737091+0000 (oldest deadline 2024-09-19T09:43:40.837238+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:42.479+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:42.479+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:42.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:42 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:42.479+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:42.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:42 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:42.209+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:42.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:42 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:42.209+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:42.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:42 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:42.209+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:42.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:42.306+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:42.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:42.306+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:43.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:42 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:42.796+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:43.169 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:42 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:42.796+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:43.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:42.933+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:43.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:42.933+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:43.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:42 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:42.933+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:43.387 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:43 smithi078 podman[71676]: 2024-09-19 09:43:43.127644564 +0000 UTC m=+5.495525590 container died 96eedc02a30558ba78e6c99e05840a2c6f19a537a9b48b9e2f25a6c322cd9201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:43:43.387 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:43.276+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:43.387 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:43.276+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:43.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:43.168+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:43.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:43.168+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:43.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:43.168+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:43.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:43.346+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:43.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:43.346+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:43.769 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:43 smithi078 podman[71676]: 2024-09-19 09:43:43.475971733 +0000 UTC m=+5.843852759 container cleanup 96eedc02a30558ba78e6c99e05840a2c6f19a537a9b48b9e2f25a6c322cd9201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:43:43.769 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:43 smithi078 bash[71676]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5 2024-09-19T09:43:43.769 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:43.468+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:43.769 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:43.468+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:43.769 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:43 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:43.468+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:44.091 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:43 smithi078 podman[71692]: 2024-09-19 09:43:43.767855837 +0000 UTC m=+0.639317745 container remove 96eedc02a30558ba78e6c99e05840a2c6f19a537a9b48b9e2f25a6c322cd9201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T09:43:44.148 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:43.750+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:44.148 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:43 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:43.750+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:44.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:43.952+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:44.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:43.952+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:44.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:43.952+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:44.356 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:44.263+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:44.356 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:44.263+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:44.357 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:43.99294528 +0000 UTC m=+0.015864352 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:44.357 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:44.111111475 +0000 UTC m=+0.134030549 container create 600f7502e5e9c9dd2bd7adc0738fb3b8bafdd84d8763d8798ea6dfc90a36207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:43:44.357 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:44.355285317 +0000 UTC m=+0.378204388 container init 600f7502e5e9c9dd2bd7adc0738fb3b8bafdd84d8763d8798ea6dfc90a36207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T09:43:44.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:44 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:44.146+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:44.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:44 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:44.146+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:44.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:44 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:44.146+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:44.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:44.437+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:44.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:44.437+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:44.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:44 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:44.437+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:44.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:44.359126032 +0000 UTC m=+0.382045104 container start 600f7502e5e9c9dd2bd7adc0738fb3b8bafdd84d8763d8798ea6dfc90a36207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T09:43:44.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:44.43859578 +0000 UTC m=+0.461514858 container attach 600f7502e5e9c9dd2bd7adc0738fb3b8bafdd84d8763d8798ea6dfc90a36207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T09:43:44.608 INFO:journalctl@ceph.osd.5.smithi078.stdout:Sep 19 09:43:44 smithi078 podman[71781]: 2024-09-19 09:43:44.551059911 +0000 UTC m=+0.573978993 container died 600f7502e5e9c9dd2bd7adc0738fb3b8bafdd84d8763d8798ea6dfc90a36207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T09:43:44.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:44.390+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:44.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:44.390+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:45.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T09:43:45.164 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:44 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:44.769+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:45.165 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:44 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:44.769+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:45.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:45.185 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T09:43:45.185 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T09:43:45.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.6 2024-09-19T09:43:45.214 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:44.946+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:45.214 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:44.946+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:45.215 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:44 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:44.946+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:45.394 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:45.265+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:45.394 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:45.265+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:45.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:45 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:45.163+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:45.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:45 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:45.163+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:45.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:45 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:45.163+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:45.500 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 systemd[1]: Stopping Ceph osd.6 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:45.500 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:45.417+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:45.500 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:45.417+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:45.500 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:45.417+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:45.764 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.497+0000 7f5402731640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:45.764 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.497+0000 7f5402731640 -1 osd.6 88 *** Got signal Terminated *** 2024-09-19T09:43:45.764 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.497+0000 7f5402731640 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:45.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:45.478+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:45.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:45.478+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:45.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:45 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:45.478+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:46.154 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:45 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:45.789+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:46.155 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:45 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:45.789+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:46.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.937+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:46.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.937+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:46.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:45 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:45.937+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:46.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:46.153+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:46.427 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:46.153+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:46.427 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:46.153+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:46.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:46.261+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:46.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:46.261+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:46.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:46.507+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:46.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:46.507+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:46.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:46 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:46.507+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:46.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:46.430+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:46.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:46.430+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:46.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:46.430+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:47.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:46.810+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:47.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:46.810+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:47.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:46.962+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:47.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:46.962+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:47.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:46 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:46.962+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:47.533 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:47.276+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:47.533 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:47.276+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:47.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:47.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:47.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:47.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:47.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:47.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:47.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:47.432+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:47.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:47.432+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:47.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:47.432+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:47.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:47.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:47.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:47.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:47.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:47 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:47.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:48.153 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:47.845+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:48.153 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:47.845+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:48.153 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:47 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:47.845+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:48.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:47.938+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:48.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:47.938+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:48.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:47 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:47.938+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:48.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:48.193+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:48.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:48.193+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:48.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:48.193+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:48.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:48.312+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:48.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:48.312+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:48.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:48.556+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:48.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:48.556+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:48.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:48 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:48.556+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:48.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:48.386+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:48.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:48.386+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:48.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:48.386+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:49.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:48.859+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:49.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:48.859+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:49.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:48.859+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:49.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:48.904+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:49.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:48.904+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:49.265 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:48 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:48.904+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:49.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:49 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:49.278+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:49.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:49 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:49.278+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:49.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:49 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:49.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:49.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:49 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:49.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:49.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:49 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:49.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:49.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:49.195+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:49.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:49.195+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:49.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:49.195+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:49.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:49.377+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:49.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:49.377+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:49.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:49.377+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:49.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:49.377+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:50.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:49.891+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:50.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:49.891+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:50.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:49 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:49.891+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:50.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:49.879+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:43:01.967462+0000 front 2024-09-19T09:43:01.967560+0000 (oldest deadline 2024-09-19T09:43:27.867551+0000) 2024-09-19T09:43:50.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:49.879+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.168605+0000 front 2024-09-19T09:43:10.168443+0000 (oldest deadline 2024-09-19T09:43:35.468602+0000) 2024-09-19T09:43:50.264 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:49 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6[58002]: 2024-09-19T09:43:49.879+0000 7f53fed36640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:15.469156+0000 front 2024-09-19T09:43:15.469184+0000 (oldest deadline 2024-09-19T09:43:41.369321+0000) 2024-09-19T09:43:50.530 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:50.270+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:50.531 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:50.270+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:50.531 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:50.270+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:50.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:50.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:50.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:50.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:50.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:50.178+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:50.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:50.424+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:50.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:50.424+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:50.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:50.424+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:50.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:50 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:50.424+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:50.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:50.528+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:50.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:50.528+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:50.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:50 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:50.528+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:51.151 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:50.860+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:51.152 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:50.860+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:51.152 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:50 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:50.860+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:51.309 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:51 smithi028 podman[103898]: 2024-09-19 09:43:51.032981724 +0000 UTC m=+5.641910284 container died b8ef7a90a2f42ea979b0739f490f8c49a3caa5b5514027f689c4560423812c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T09:43:51.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:51.150+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:51.435 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:51.150+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:51.435 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:51.150+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:51.435 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:51.150+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:51.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:51.273+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:51.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:51.273+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:51.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:51.273+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:51 smithi028 podman[103898]: 2024-09-19 09:43:51.398430392 +0000 UTC m=+6.007358941 container cleanup b8ef7a90a2f42ea979b0739f490f8c49a3caa5b5514027f689c4560423812c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:51 smithi028 bash[103898]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:51.442+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:51.442+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:51.442+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:51.676 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:51 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:51.442+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:51.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:51.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:51.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:51.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:51.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:51 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:51.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:52.014 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:51 smithi028 podman[103915]: 2024-09-19 09:43:51.674963248 +0000 UTC m=+0.642831400 container remove b8ef7a90a2f42ea979b0739f490f8c49a3caa5b5514027f689c4560423812c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T09:43:52.141 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:51.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:52.141 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:51.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:52.141 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:51 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:51.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:52.329 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:52.025803366 +0000 UTC m=+0.113554767 container create d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:43:52.329 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:51.928946893 +0000 UTC m=+0.016698264 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:52.329 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:52.228087297 +0000 UTC m=+0.315838657 container init d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T09:43:52.329 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:52.231533315 +0000 UTC m=+0.319284675 container start d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T09:43:52.329 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:52.327986204 +0000 UTC m=+0.415737566 container attach d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T09:43:52.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:52.140+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:52.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:52.140+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:52.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:52.140+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:52.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:52.140+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:52.513 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:52.231+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:52.513 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:52.231+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:52.513 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:52.231+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:52.716 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104001]: 2024-09-19 09:43:52.425747507 +0000 UTC m=+0.513498879 container died d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T09:43:52.716 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:52 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:52.486+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:52.716 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:52 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:52.486+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:52.716 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:52 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:52.486+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:52.716 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:52 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:52.486+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:52.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:52.511+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:52.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:52.511+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:52.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:52 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:52.511+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:52.967 INFO:journalctl@ceph.osd.6.smithi028.stdout:Sep 19 09:43:52 smithi028 podman[104024]: 2024-09-19 09:43:52.9483126 +0000 UTC m=+0.515840523 container remove d74f005eaa67800576e01b2011ff6e2fc71f6eb2632b077a1f32ae7ba77e6853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:43:52.972 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T09:43:53.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:43:53.056 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T09:43:53.056 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T09:43:53.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.7 2024-09-19T09:43:53.084 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:52.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:53.084 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:52.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:53.084 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:52.827+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:53.353 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:53.137+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:53.353 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:53.137+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:53.353 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:53.137+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:53.353 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:53.137+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:53.354 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 systemd[1]: Stopping Ceph osd.7 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:43:53.354 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.351+0000 7fc9488ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:43:53.354 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.351+0000 7fc9488ee640 -1 osd.7 88 *** Got signal Terminated *** 2024-09-19T09:43:53.354 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.351+0000 7fc9488ee640 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:43:53.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:53.257+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:53.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:53.257+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:53.543 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:53.257+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:53.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:53.467+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:53.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:53.467+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:53.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:53.467+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:53.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:53 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:53.467+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:53.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:53.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:53.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:53.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:53.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:53 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:53.541+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:54.107 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.780+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:54.107 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.780+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:54.107 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:53 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:53.780+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:54.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:54.106+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:54.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:54.106+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:54.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:54.106+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:54.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:54.106+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:54.515 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:54.252+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:54.515 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:54.252+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:54.515 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:54.252+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:54.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:54 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:54.469+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:54.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:54 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:54.469+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:54.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:54 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:54.469+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:54.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:54 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:54.469+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:54.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:54.513+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:54.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:54.513+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:54.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:54 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:54.513+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:55.146 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:54.831+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:55.146 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:54.831+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:55.146 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:54 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:54.831+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:55.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:55.145+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:55.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:55.145+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:55.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:55.145+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:55.426 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:55.145+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:55.532 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:55.237+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:55.532 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:55.237+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:55.533 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:55.237+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:55.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:55.435+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:55.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:55.435+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:55.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:55.435+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:55.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:55 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:55.435+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:55.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:55.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:55.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:55.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:55.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:55 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:55.531+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:56.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:55.794+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:56.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:55.794+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:56.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:55 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:55.794+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:56.282+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:56.282+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:56.282+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:56.489+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:56.489+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:56.489+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:56.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:56 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:56.489+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:43:56.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:56.182+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:56.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:56.182+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:56.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:56.182+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:56.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:56.182+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:56.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:56 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:56.450+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:56.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:56 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:56.450+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:56.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:56 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:56.450+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:56.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:56 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:56.450+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:57.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:56.796+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:57.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:56.796+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:57.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:56.796+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:57.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:57.292+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:57.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:57.293+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:57.607 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:57.293+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:57.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:57.293+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:43:57.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:57.493+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:57.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:57.493+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:57.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:57.493+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:57.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:57 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:57.493+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:43:57.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:57.223+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:57.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:57.223+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:57.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:57.223+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:57.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:57.223+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:57.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:57.223+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:43:57.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:57.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:57.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:57.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:57.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:57.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:57.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:57 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:57.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:58.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:57.839+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.502568+0000 front 2024-09-19T09:43:10.502540+0000 (oldest deadline 2024-09-19T09:43:36.402673+0000) 2024-09-19T09:43:58.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:57.839+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.403078+0000 front 2024-09-19T09:43:16.403173+0000 (oldest deadline 2024-09-19T09:43:41.703151+0000) 2024-09-19T09:43:58.176 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:57 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7[53491]: 2024-09-19T09:43:57.839+0000 7fc940eeb640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:21.704000+0000 front 2024-09-19T09:43:21.703980+0000 (oldest deadline 2024-09-19T09:43:47.603723+0000) 2024-09-19T09:43:58.518 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:58.247+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:58.518 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:58.247+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:58.518 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:58.247+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:58.518 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:58.247+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:43:58.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:58 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:58.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:58.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:58 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:58.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:58.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:58 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:58.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:58.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:58 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:58.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:58.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:58 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:58.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:43:58.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:58.478+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:58.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:58.478+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:58.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:58.478+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:58.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:58.478+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:58.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:58 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:58.478+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:43:58.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:58.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:58.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:58.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:58.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:58.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:58.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:58 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:58.516+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:43:58.987 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:58 smithi038 podman[70680]: 2024-09-19 09:43:58.706229244 +0000 UTC m=+5.447161704 container died cf018c9ecf7bf88bb81d86ebba275452ca330cea7f63963c16410a15d237ad60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T09:43:59.275 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70680]: 2024-09-19 09:43:59.068274687 +0000 UTC m=+5.809207148 container cleanup cf018c9ecf7bf88bb81d86ebba275452ca330cea7f63963c16410a15d237ad60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:43:59.275 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 bash[70680]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7 2024-09-19T09:43:59.488 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:59.218+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:43:59.489 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:59.218+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:43:59.489 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:59.218+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:43:59.489 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:43:59.218+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:43:59.671 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70695]: 2024-09-19 09:43:59.352694801 +0000 UTC m=+0.644942930 container remove cf018c9ecf7bf88bb81d86ebba275452ca330cea7f63963c16410a15d237ad60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:43:59.671 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:59 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:59.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:43:59.671 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:59 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:59.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:43:59.671 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:59 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:59.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:43:59.672 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:59 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:59.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:43:59.672 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:43:59 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:43:59.272+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:43:59.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:59.497+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:43:59.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:59.497+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:43:59.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:59.497+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:43:59.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:59.497+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:43:59.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:43:59 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:43:59.497+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:43:59.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:59.485+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:43:59.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:59.485+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:43:59.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:59.485+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:43:59.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:43:59 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:43:59.485+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:43:59.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70783]: 2024-09-19 09:43:59.670088069 +0000 UTC m=+0.104077777 container create 435b3a5c794ad52cebd35f3b4153b4e8cc7855dac888e0fc346fc5333731fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:43:59.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70783]: 2024-09-19 09:43:59.581707735 +0000 UTC m=+0.015697451 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:43:59.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70783]: 2024-09-19 09:43:59.880722224 +0000 UTC m=+0.314711942 container init 435b3a5c794ad52cebd35f3b4153b4e8cc7855dac888e0fc346fc5333731fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:43:59.926 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70783]: 2024-09-19 09:43:59.883865662 +0000 UTC m=+0.317855371 container start 435b3a5c794ad52cebd35f3b4153b4e8cc7855dac888e0fc346fc5333731fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T09:44:00.314 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:43:59 smithi038 podman[70783]: 2024-09-19 09:43:59.9641622 +0000 UTC m=+0.398151925 container attach 435b3a5c794ad52cebd35f3b4153b4e8cc7855dac888e0fc346fc5333731fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:44:00.314 INFO:journalctl@ceph.osd.7.smithi038.stdout:Sep 19 09:44:00 smithi038 podman[70783]: 2024-09-19 09:44:00.079336458 +0000 UTC m=+0.513326179 container died 435b3a5c794ad52cebd35f3b4153b4e8cc7855dac888e0fc346fc5333731fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T09:44:00.581 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:00 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:00.310+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:00.581 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:00 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:00.310+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:00.581 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:00 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:00.310+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:00.581 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:00 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:00.310+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:00.581 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:00 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:00.310+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:00.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:00.442+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:00.442+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:00.442+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:00.442+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:00.201+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:00.201+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:00.201+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:00.608 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:00.201+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:00.694 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T09:44:00.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:00.475+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:00.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:00.475+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:00.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:00.475+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:00.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:00.475+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:00.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:00 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:00.475+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:00.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:00.772 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T09:44:00.773 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T09:44:00.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.8 2024-09-19T09:44:01.088 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:00 smithi078 systemd[1]: Stopping Ceph osd.8 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:01.357 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.086+0000 7f79e6528640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.086+0000 7f79e6528640 -1 osd.8 88 *** Got signal Terminated *** 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.086+0000 7f79e6528640 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.163+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.163+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.163+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:01.358 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:01.163+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:01.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:01 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:01.338+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:01.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:01 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:01.338+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:01.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:01 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:01.338+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:01.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:01 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:01.338+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:01.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:01 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:01.338+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:01.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:01 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:01.441+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:01.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:01 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:01.441+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:01.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:01 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:01.441+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:01.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:01 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:01.441+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:01.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:01 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:01.441+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:01.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:01.473+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:01.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:01.473+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:01.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:01.473+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:01.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:01 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:01.473+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:02.525 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:02.154+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:02.525 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:02.154+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:02.525 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:02.154+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:02.525 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:02.154+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:02.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:02 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:02.300+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:02.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:02 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:02.300+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:02.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:02 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:02.300+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:02.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:02 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:02.300+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:02.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:02 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:02.300+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:02.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:02.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:02.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:02.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:02.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:02.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:02.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:02.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:02.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:02 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:02.473+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:02.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:02.523+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:02.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:02.523+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:02.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:02.523+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:02.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:02 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:02.523+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:03.483 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:03.158+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:03.483 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:03.158+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:03.483 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:03.158+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:03.483 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:03.158+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:03.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:03 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:03.254+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:03.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:03 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:03.254+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:03.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:03 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:03.254+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:03.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:03 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:03.254+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:03.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:03 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:03.254+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:03.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:03 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:03.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:03.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:03 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:03.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:03.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:03 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:03.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:03.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:03 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:03.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:03.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:03 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:03.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:03.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:03.481+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:03.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:03.481+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:03.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:03.481+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:03.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:03 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:03.481+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:04.470 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:04.147+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:04.471 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:04.147+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:04.471 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:04.147+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:04.471 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:04.147+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:04.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:04.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:04.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:04.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:04.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:04.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:04 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:04.291+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:04.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:04.469+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:04.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:04.469+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:04.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:04.469+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:04.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:04.469+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:05.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:05.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:05.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:05.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:05.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:05.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:04 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:04.517+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:05.476 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:05.111+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.805348+0000 front 2024-09-19T09:43:10.805151+0000 (oldest deadline 2024-09-19T09:43:34.305311+0000) 2024-09-19T09:44:05.476 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:05.111+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:17.106625+0000 front 2024-09-19T09:43:17.106716+0000 (oldest deadline 2024-09-19T09:43:40.507323+0000) 2024-09-19T09:44:05.476 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:05.111+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.608304+0000 front 2024-09-19T09:43:24.608428+0000 (oldest deadline 2024-09-19T09:43:49.308367+0000) 2024-09-19T09:44:05.476 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8[54297]: 2024-09-19T09:44:05.111+0000 7f79deb25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.609469+0000 front 2024-09-19T09:43:31.609356+0000 (oldest deadline 2024-09-19T09:43:56.810026+0000) 2024-09-19T09:44:05.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.312+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:05.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.312+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:05.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.312+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:05.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.312+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:05.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.313+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:05.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:05 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:05.313+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:05.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:05.474+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:05.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:05.474+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:05.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:05.474+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:05.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:05 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:05.474+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:06.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:06.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:06.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:06.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:06.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:06.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:05 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:05.513+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:06.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:06.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:06.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:06.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:06.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:06.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:06 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:06.294+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:06.755 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:06 smithi078 podman[71895]: 2024-09-19 09:44:06.42512033 +0000 UTC m=+5.455289785 container died 6d3635534e64f939e7ad02c8871f9243f13acb6e3dd1e002eb37ccb84d8223d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T09:44:06.756 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:06 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:06.522+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:06.757 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:06 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:06.522+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:06.757 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:06 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:06.522+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:06.757 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:06 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:06.522+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:06.757 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:06 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:06.522+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:06.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:06.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:06.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:06.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:06.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:06.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:06 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:06.500+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:07.063 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:06 smithi078 podman[71895]: 2024-09-19 09:44:06.860773018 +0000 UTC m=+5.890942479 container cleanup 6d3635534e64f939e7ad02c8871f9243f13acb6e3dd1e002eb37ccb84d8223d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T09:44:07.064 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:06 smithi078 bash[71895]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8 2024-09-19T09:44:07.357 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71911]: 2024-09-19 09:44:07.061891878 +0000 UTC m=+0.629519389 container remove 6d3635534e64f939e7ad02c8871f9243f13acb6e3dd1e002eb37ccb84d8223d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:44:07.589 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:07.590 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:07.590 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:07.590 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:07.590 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:07.590 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:07 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:07.311+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:07.741 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:07 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:07.558+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:07.741 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:07 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:07.558+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:07.741 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:07 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:07.558+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:07.741 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:07 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:07.558+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:07.741 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:07 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:07.558+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:07.742 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.470905918 +0000 UTC m=+0.112229452 container create 479752f44b6cf2e869547fad0622825dd480123f6da47d6ce117d750b4472548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:44:07.742 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.374387655 +0000 UTC m=+0.015711198 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:44:07.742 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.681625968 +0000 UTC m=+0.322949502 container init 479752f44b6cf2e869547fad0622825dd480123f6da47d6ce117d750b4472548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:44:07.742 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.684797911 +0000 UTC m=+0.326121455 container start 479752f44b6cf2e869547fad0622825dd480123f6da47d6ce117d750b4472548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:44:07.742 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.73968912 +0000 UTC m=+0.381012669 container attach 479752f44b6cf2e869547fad0622825dd480123f6da47d6ce117d750b4472548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True) 2024-09-19T09:44:08.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:08.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:08.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:08.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:08.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:08.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:07 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:07.526+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:08.107 INFO:journalctl@ceph.osd.8.smithi078.stdout:Sep 19 09:44:07 smithi078 podman[71999]: 2024-09-19 09:44:07.877198738 +0000 UTC m=+0.518522282 container died 479752f44b6cf2e869547fad0622825dd480123f6da47d6ce117d750b4472548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:44:08.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T09:44:08.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:08.542 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T09:44:08.542 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T09:44:08.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.9 2024-09-19T09:44:08.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:08.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:08.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:08.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:08.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:08.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:08 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:08.290+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:08.750 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:08.596+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:08.751 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:08.596+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:08.751 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:08.596+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:08.751 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:08.596+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:08.751 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:08.596+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.522+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:08.853 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 systemd[1]: Stopping Ceph osd.9 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:09.264 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.851+0000 7f803b6b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:09.264 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.851+0000 7f803b6b3640 -1 osd.9 88 *** Got signal Terminated *** 2024-09-19T09:44:09.264 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:08 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:08.851+0000 7f803b6b3640 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:09.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:09 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:09.282+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:09.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:09.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:09.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:09.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:09.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:09.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:09 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:09.477+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:10.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:09 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:09.637+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:10.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:09 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:09.637+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:10.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:09 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:09.637+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:10.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:09 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:09.637+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:10.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:09 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:09.637+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:10.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:10 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:10.299+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:10.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:10.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:10.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:10.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:10.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:10.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:10 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:10.506+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:11.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:10.685+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:11.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:10.685+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:11.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:10.685+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:11.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:10.685+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:11.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:10 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:10.685+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:11.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:11 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:11.348+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:11.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:11.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:11.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:11.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:11.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:11.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:11 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:11.488+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:12.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:11 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:11.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:12.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:11 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:11.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:12.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:11 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:11.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:12.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:11 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:11.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:12.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:11 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:11.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:12.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:12 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:12.380+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:13.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:12 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:12.532+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.873194+0000 front 2024-09-19T09:43:46.873250+0000 (oldest deadline 2024-09-19T09:44:12.173201+0000) 2024-09-19T09:44:13.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:13.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:13.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:13.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:13.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:13.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:12.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:13.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:13.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:13.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:13.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:13.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:13.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:13.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:13 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:13.344+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:13.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:58.163749+0000 front 2024-09-19T09:42:58.163706+0000 (oldest deadline 2024-09-19T09:43:24.064087+0000) 2024-09-19T09:44:13.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:10.966401+0000 front 2024-09-19T09:43:10.966351+0000 (oldest deadline 2024-09-19T09:43:33.167026+0000) 2024-09-19T09:44:13.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:18.869748+0000 front 2024-09-19T09:43:18.869759+0000 (oldest deadline 2024-09-19T09:43:44.769551+0000) 2024-09-19T09:44:13.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:24.770067+0000 front 2024-09-19T09:43:24.769964+0000 (oldest deadline 2024-09-19T09:43:48.869999+0000) 2024-09-19T09:44:13.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.071563+0000 front 2024-09-19T09:43:34.071449+0000 (oldest deadline 2024-09-19T09:43:58.171363+0000) 2024-09-19T09:44:13.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:38.172011+0000 front 2024-09-19T09:43:38.171880+0000 (oldest deadline 2024-09-19T09:44:04.071835+0000) 2024-09-19T09:44:13.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:13 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9[64548]: 2024-09-19T09:44:13.502+0000 7f8033cb0640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.873194+0000 front 2024-09-19T09:43:46.873250+0000 (oldest deadline 2024-09-19T09:44:12.173201+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:14.009 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:13 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:13.692+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:14.502 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:14 smithi028 podman[104117]: 2024-09-19 09:44:14.202088932 +0000 UTC m=+5.456558398 container died 0a0908ae4a92d0754c369c981ff2c8e8bc7b3e868258c5d1c91be9ad438db2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T09:44:14.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:14.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:14.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:14.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:14.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:14.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:14.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:14 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:14.336+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:14.764 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:14 smithi028 podman[104117]: 2024-09-19 09:44:14.600187055 +0000 UTC m=+5.854656512 container cleanup 0a0908ae4a92d0754c369c981ff2c8e8bc7b3e868258c5d1c91be9ad438db2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T09:44:14.765 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:14 smithi028 bash[104117]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9 2024-09-19T09:44:15.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:15.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:15.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:15.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:15.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:15.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:14 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:14.675+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:15.264 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:14 smithi028 podman[104134]: 2024-09-19 09:44:14.901573545 +0000 UTC m=+0.694468838 container remove 0a0908ae4a92d0754c369c981ff2c8e8bc7b3e868258c5d1c91be9ad438db2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T09:44:15.665 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.360653413 +0000 UTC m=+0.105134579 container create f57fe66941b7e41739a4679fa0b5bc93e33449a77abc47a1623faa430376d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T09:44:15.665 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.272218808 +0000 UTC m=+0.016699976 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:44:15.665 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.579709836 +0000 UTC m=+0.324191013 container init f57fe66941b7e41739a4679fa0b5bc93e33449a77abc47a1623faa430376d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T09:44:15.665 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.583071494 +0000 UTC m=+0.327552662 container start f57fe66941b7e41739a4679fa0b5bc93e33449a77abc47a1623faa430376d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:15.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:15.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:15 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:15.326+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:16.014 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.662979193 +0000 UTC m=+0.407460374 container attach f57fe66941b7e41739a4679fa0b5bc93e33449a77abc47a1623faa430376d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T09:44:16.015 INFO:journalctl@ceph.osd.9.smithi028.stdout:Sep 19 09:44:15 smithi028 podman[104220]: 2024-09-19 09:44:15.779876602 +0000 UTC m=+0.524357778 container died f57fe66941b7e41739a4679fa0b5bc93e33449a77abc47a1623faa430376d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:44:16.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:16.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:16.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:16.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:16.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:16.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:15 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:15.712+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:16.503 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T09:44:16.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:16.549 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T09:44:16.549 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T09:44:16.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.10 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:16.577 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:16.578 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.305+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:16.838 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 systemd[1]: Stopping Ceph osd.10 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:17.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:17.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:17.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:17.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:17.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:17.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:16 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:16.718+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:17.176 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.836+0000 7f2408d9e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:17.176 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.836+0000 7f2408d9e640 -1 osd.10 88 *** Got signal Terminated *** 2024-09-19T09:44:17.176 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:16 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:16.836+0000 7f2408d9e640 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:44:17.633 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:17.633 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:17.633 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:17.633 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:17.633 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:17.634 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:17.634 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:17 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:17.325+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:18.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:18.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:18.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:18.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:18.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:18.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:17 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:17.734+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:18.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:18.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:18 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:18.339+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:19.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:19.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:19.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:19.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:19.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:19.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:18 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:18.781+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:19.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:19.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:19.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:19.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:19.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:19.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:19.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:19 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:19.389+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:20.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:20.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:20.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:20.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:20.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:20.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:19 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:19.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:20.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:20.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:20.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:20.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:20.927 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:20.927 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:20.927 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:20 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:20.437+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:21.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:21.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:21.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:21.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:21.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:21.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:20 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:20.756+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:21.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.730956+0000 front 2024-09-19T09:42:59.730869+0000 (oldest deadline 2024-09-19T09:43:25.630958+0000) 2024-09-19T09:44:21.676 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:09.632494+0000 front 2024-09-19T09:43:09.632615+0000 (oldest deadline 2024-09-19T09:43:35.532724+0000) 2024-09-19T09:44:21.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.633787+0000 front 2024-09-19T09:43:16.634002+0000 (oldest deadline 2024-09-19T09:43:40.733780+0000) 2024-09-19T09:44:21.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6814 osd.3 since back 2024-09-19T09:43:26.634993+0000 front 2024-09-19T09:43:26.635025+0000 (oldest deadline 2024-09-19T09:43:50.735063+0000) 2024-09-19T09:44:21.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:34.136367+0000 front 2024-09-19T09:43:34.136419+0000 (oldest deadline 2024-09-19T09:43:56.436574+0000) 2024-09-19T09:44:21.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:37.537780+0000 front 2024-09-19T09:43:37.537578+0000 (oldest deadline 2024-09-19T09:44:03.437457+0000) 2024-09-19T09:44:21.677 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:21 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10[60233]: 2024-09-19T09:44:21.417+0000 7f240139b640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:49.238985+0000 front 2024-09-19T09:43:49.238854+0000 (oldest deadline 2024-09-19T09:44:12.738669+0000) 2024-09-19T09:44:22.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:22.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:22.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:22.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:22.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:22.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:21 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:21.730+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:22.605 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:22 smithi038 podman[70899]: 2024-09-19 09:44:22.341365407 +0000 UTC m=+5.595209249 container died c34e86b861169a532a566f0cfdc1a564fefe8e9d0330d9d7d9148ef1db3fdb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T09:44:22.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:22 smithi038 podman[70899]: 2024-09-19 09:44:22.686337239 +0000 UTC m=+5.940181071 container cleanup c34e86b861169a532a566f0cfdc1a564fefe8e9d0330d9d7d9148ef1db3fdb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:44:22.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:22 smithi038 bash[70899]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10 2024-09-19T09:44:22.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:22 smithi038 podman[70914]: 2024-09-19 09:44:22.84598951 +0000 UTC m=+0.497694797 container remove c34e86b861169a532a566f0cfdc1a564fefe8e9d0330d9d7d9148ef1db3fdb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T09:44:23.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:23.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:22 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:22.744+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:23.550 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.155642479 +0000 UTC m=+0.015816515 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:44:23.550 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.254932752 +0000 UTC m=+0.115106776 container create 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T09:44:23.550 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.47421772 +0000 UTC m=+0.334391746 container init 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T09:44:23.550 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.477354133 +0000 UTC m=+0.337528157 container start 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T09:44:23.550 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.549092256 +0000 UTC m=+0.409266287 container attach 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T09:44:23.926 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:23 smithi038 podman[71004]: 2024-09-19 09:44:23.668814865 +0000 UTC m=+0.528988895 container died 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:44:24.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:24.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:24.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:24.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:24.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:24.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:24.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:23 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:23.747+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:24.220 INFO:journalctl@ceph.osd.10.smithi038.stdout:Sep 19 09:44:24 smithi038 podman[71027]: 2024-09-19 09:44:24.194051216 +0000 UTC m=+0.514035793 container remove 5c7212eac5da8fc8b27cd6001dfa5fd23ad027d875210dd0ffa7a948fb4c4a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:44:24.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T09:44:24.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:24.304 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T09:44:24.304 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T09:44:24.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.11 2024-09-19T09:44:24.589 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 systemd[1]: Stopping Ceph osd.11 for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:24.589 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.586+0000 7f5cfa744640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:24.589 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.586+0000 7f5cfa744640 -1 osd.11 88 *** Got signal Terminated *** 2024-09-19T09:44:24.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.586+0000 7f5cfa744640 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:44:24.857 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:24.858 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:24 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:24.790+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:26.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:26.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:26.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:26.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:26.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:26.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:26.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:25 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:25.780+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:27.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:27.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:26 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:26.739+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:28.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:28.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:27 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:27.728+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6820 osd.8 since back 2024-09-19T09:44:02.303290+0000 front 2024-09-19T09:44:02.303332+0000 (oldest deadline 2024-09-19T09:44:27.503998+0000) 2024-09-19T09:44:29.107 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6806 osd.0 since back 2024-09-19T09:42:59.393610+0000 front 2024-09-19T09:42:59.393556+0000 (oldest deadline 2024-09-19T09:43:25.293724+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6804 osd.1 since back 2024-09-19T09:43:11.095366+0000 front 2024-09-19T09:43:11.095442+0000 (oldest deadline 2024-09-19T09:43:32.695829+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6804 osd.2 since back 2024-09-19T09:43:16.097121+0000 front 2024-09-19T09:43:16.097140+0000 (oldest deadline 2024-09-19T09:43:41.397323+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6812 osd.4 since back 2024-09-19T09:43:31.799923+0000 front 2024-09-19T09:43:31.799986+0000 (oldest deadline 2024-09-19T09:43:55.899919+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6812 osd.5 since back 2024-09-19T09:43:41.200717+0000 front 2024-09-19T09:43:41.200776+0000 (oldest deadline 2024-09-19T09:44:06.500915+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.28:6822 osd.6 since back 2024-09-19T09:43:46.501420+0000 front 2024-09-19T09:43:46.501503+0000 (oldest deadline 2024-09-19T09:44:11.801303+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.38:6820 osd.7 since back 2024-09-19T09:43:57.603084+0000 front 2024-09-19T09:43:57.603026+0000 (oldest deadline 2024-09-19T09:44:22.302946+0000) 2024-09-19T09:44:29.108 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:28 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11[61053]: 2024-09-19T09:44:28.770+0000 7f5cf6d49640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.78:6820 osd.8 since back 2024-09-19T09:44:02.303290+0000 front 2024-09-19T09:44:02.303332+0000 (oldest deadline 2024-09-19T09:44:27.503998+0000) 2024-09-19T09:44:30.347 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:30 smithi078 podman[72114]: 2024-09-19 09:44:30.031906951 +0000 UTC m=+5.533559554 container died eea0baf1c7f5293b826b6d02f23b4aed34cd97e88c5f1f0c41720162b87df951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T09:44:30.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:30 smithi078 podman[72114]: 2024-09-19 09:44:30.445290767 +0000 UTC m=+5.946943366 container cleanup eea0baf1c7f5293b826b6d02f23b4aed34cd97e88c5f1f0c41720162b87df951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T09:44:30.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:30 smithi078 bash[72114]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11 2024-09-19T09:44:31.077 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:30 smithi078 podman[72128]: 2024-09-19 09:44:30.671296221 +0000 UTC m=+0.636279602 container remove eea0baf1c7f5293b826b6d02f23b4aed34cd97e88c5f1f0c41720162b87df951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:44:31.329 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:30.979079238 +0000 UTC m=+0.015526344 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:44:31.329 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:31.080114916 +0000 UTC m=+0.116562010 container create f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:44:31.329 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:31.324158938 +0000 UTC m=+0.360606031 container init f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T09:44:31.329 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:31.327533779 +0000 UTC m=+0.363980875 container start f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T09:44:31.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:31.432897592 +0000 UTC m=+0.469344686 container attach f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T09:44:31.607 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:31 smithi078 podman[72212]: 2024-09-19 09:44:31.509815869 +0000 UTC m=+0.546262963 container died f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T09:44:32.202 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T09:44:32.226 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:32 smithi078 podman[72235]: 2024-09-19 09:44:32.177499245 +0000 UTC m=+0.659867809 container remove f06229433db0973ab8244e2ac73bd33d69341bb7ab5db6751898844c470299e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:44:32.227 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:32 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-19T09:44:32.227 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:32 smithi078 systemd[1]: Stopped Ceph osd.11 for 9a2e9aa4-7668-11ef-bceb-c7b262605968. 2024-09-19T09:44:32.227 INFO:journalctl@ceph.osd.11.smithi078.stdout:Sep 19 09:44:32 smithi078 systemd[1]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@osd.11.service: Consumed 1min 5.041s CPU time. 2024-09-19T09:44:32.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:32.243 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T09:44:32.244 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T09:44:32.244 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.a 2024-09-19T09:44:32.536 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:32 smithi028 systemd[1]: Stopping Ceph mds.a for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:33.014 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:32 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: 2024-09-19T09:44:32.533+0000 7f3fe49bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:33.014 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:32 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a[71602]: 2024-09-19T09:44:32.533+0000 7f3fe49bd640 -1 mds.a *** got signal Terminated *** 2024-09-19T09:44:35.444 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:35 smithi028 podman[104321]: 2024-09-19 09:44:35.064410827 +0000 UTC m=+2.622393094 container died 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:44:35.764 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:35 smithi028 podman[104321]: 2024-09-19 09:44:35.5423964 +0000 UTC m=+3.100378654 container cleanup 71bbc033a836031d5d55ab669beb10b9a682b1a7ff3a9d741d51e47cbee7bed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T09:44:35.764 INFO:journalctl@ceph.mds.a.smithi028.stdout:Sep 19 09:44:35 smithi028 bash[104321]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-a 2024-09-19T09:44:35.941 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T09:44:35.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:35.976 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T09:44:35.976 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T09:44:35.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d 2024-09-19T09:44:36.264 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:36 smithi028 systemd[1]: Stopping Ceph mds.d for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:36.764 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[100591]: 2024-09-19T09:44:36.387+0000 7fe36a239640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:36.764 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:36 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d[100591]: 2024-09-19T09:44:36.387+0000 7fe36a239640 -1 mds.d *** got signal Terminated *** 2024-09-19T09:44:37.764 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:37 smithi028 podman[104442]: 2024-09-19 09:44:37.482719323 +0000 UTC m=+1.291253076 container died 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T09:44:38.132 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:37 smithi028 podman[104442]: 2024-09-19 09:44:37.921013126 +0000 UTC m=+1.729546872 container cleanup 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:44:38.132 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:37 smithi028 bash[104442]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d 2024-09-19T09:44:38.397 INFO:journalctl@ceph.mds.d.smithi028.stdout:Sep 19 09:44:38 smithi028 podman[104459]: 2024-09-19 09:44:38.130566056 +0000 UTC m=+0.649625821 container remove 6f333bb3f3da450ad268898b94b2191d4ea8d6174a25083c857c1d5da986cbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T09:44:38.413 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T09:44:38.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:38.450 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T09:44:38.450 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T09:44:38.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.g 2024-09-19T09:44:38.764 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:38 smithi028 systemd[1]: Stopping Ceph mds.g for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:39.264 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: 2024-09-19T09:44:38.765+0000 7fe3c34c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:39.264 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:38 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g[71183]: 2024-09-19T09:44:38.765+0000 7fe3c34c3640 -1 mds.g *** got signal Terminated *** 2024-09-19T09:44:43.059 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:42 smithi028 podman[104578]: 2024-09-19 09:44:42.779562494 +0000 UTC m=+4.104766168 container died deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:44:43.059 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:43 smithi028 podman[104578]: 2024-09-19 09:44:43.057991998 +0000 UTC m=+4.383195714 container cleanup deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T09:44:43.431 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:43 smithi028 bash[104578]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g 2024-09-19T09:44:43.431 INFO:journalctl@ceph.mds.g.smithi028.stdout:Sep 19 09:44:43 smithi028 podman[104590]: 2024-09-19 09:44:43.225738562 +0000 UTC m=+0.442732181 container remove deb5bd5028b30a06e010115c5672042935cc43dea86854cc7ebfd4247e5b6e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T09:44:43.449 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T09:44:43.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:43.484 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T09:44:43.484 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T09:44:43.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.j 2024-09-19T09:44:43.764 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:43 smithi028 systemd[1]: Stopping Ceph mds.j for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:44.264 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j[70016]: 2024-09-19T09:44:43.795+0000 7f437fcdd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:44.265 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:43 smithi028 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j[70016]: 2024-09-19T09:44:43.795+0000 7f437fcdd640 -1 mds.j *** got signal Terminated *** 2024-09-19T09:44:45.239 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:44 smithi028 podman[104698]: 2024-09-19 09:44:44.95570703 +0000 UTC m=+1.245296924 container died 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T09:44:45.514 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:45 smithi028 podman[104698]: 2024-09-19 09:44:45.312161298 +0000 UTC m=+1.601751126 container cleanup 5b494458b16d346a8ea1ea48a6257a4cd75fa893187576f91768655af32d0660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:44:45.514 INFO:journalctl@ceph.mds.j.smithi028.stdout:Sep 19 09:44:45 smithi028 bash[104698]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-j 2024-09-19T09:44:45.731 DEBUG:teuthology.orchestra.run.smithi028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T09:44:45.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:45.765 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T09:44:45.765 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T09:44:45.765 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.b 2024-09-19T09:44:46.072 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:45 smithi038 systemd[1]: Stopping Ceph mds.b for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:46.426 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: 2024-09-19T09:44:46.068+0000 7fc7320e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:46.426 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:46 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b[65911]: 2024-09-19T09:44:46.068+0000 7fc7320e6640 -1 mds.b *** got signal Terminated *** 2024-09-19T09:44:47.339 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:47 smithi038 podman[71106]: 2024-09-19 09:44:47.08301989 +0000 UTC m=+1.113785280 container died e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:44:47.676 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:47 smithi038 podman[71106]: 2024-09-19 09:44:47.436291813 +0000 UTC m=+1.467057194 container cleanup e5d3a1618d5ec65c1bf3ca83467ccfbd6351af3c681326554a009005f0fe315d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T09:44:47.676 INFO:journalctl@ceph.mds.b.smithi038.stdout:Sep 19 09:44:47 smithi038 bash[71106]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-b 2024-09-19T09:44:47.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T09:44:47.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:47.944 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T09:44:47.944 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T09:44:47.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.e 2024-09-19T09:44:48.300 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:48 smithi038 systemd[1]: Stopping Ceph mds.e for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:48.676 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: 2024-09-19T09:44:48.297+0000 7feb1b795640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:48.676 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:48 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e[65290]: 2024-09-19T09:44:48.297+0000 7feb1b795640 -1 mds.e *** got signal Terminated *** 2024-09-19T09:44:51.351 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:51 smithi038 podman[71239]: 2024-09-19 09:44:51.083521418 +0000 UTC m=+2.907925843 container died 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:44:51.676 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:51 smithi038 podman[71239]: 2024-09-19 09:44:51.565466176 +0000 UTC m=+3.389870598 container cleanup 2d15cc925a4c3dbbaa1f14ecacdd48f225fa2a4be3592a5d1946888dac3211a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:44:51.676 INFO:journalctl@ceph.mds.e.smithi038.stdout:Sep 19 09:44:51 smithi038 bash[71239]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-e 2024-09-19T09:44:51.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T09:44:51.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:51.975 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T09:44:51.975 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T09:44:51.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.h 2024-09-19T09:44:52.426 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:52 smithi038 systemd[1]: Stopping Ceph mds.h for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:52.426 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:44:52.310+0000 7f6b5421e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:52.426 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:52 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h[65594]: 2024-09-19T09:44:52.310+0000 7f6b5421e640 -1 mds.h *** got signal Terminated *** 2024-09-19T09:44:55.362 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:55 smithi038 podman[71360]: 2024-09-19 09:44:55.087796209 +0000 UTC m=+2.885270177 container died fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:44:55.676 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:55 smithi038 podman[71360]: 2024-09-19 09:44:55.494100419 +0000 UTC m=+3.291574386 container cleanup fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T09:44:55.676 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:55 smithi038 bash[71360]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h 2024-09-19T09:44:55.975 INFO:journalctl@ceph.mds.h.smithi038.stdout:Sep 19 09:44:55 smithi038 podman[71372]: 2024-09-19 09:44:55.720791563 +0000 UTC m=+0.634449621 container remove fdff1c820477f1e7358c46f539d30c8d412f6176c31687b17607cecd72980f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T09:44:55.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T09:44:56.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:56.011 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T09:44:56.012 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T09:44:56.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.k 2024-09-19T09:44:56.333 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:56 smithi038 systemd[1]: Stopping Ceph mds.k for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:44:56.676 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: 2024-09-19T09:44:56.331+0000 7f3f8b8dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:44:56.676 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:56 smithi038 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k[64978]: 2024-09-19T09:44:56.331+0000 7f3f8b8dd640 -1 mds.k *** got signal Terminated *** 2024-09-19T09:44:59.349 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:59 smithi038 podman[71481]: 2024-09-19 09:44:59.085980323 +0000 UTC m=+2.853110134 container died 53d58698adb10f08801b1389abdcd2bfdaf1b8ba10f1ef50cc3f50a4a94c9f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T09:44:59.676 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:59 smithi038 podman[71481]: 2024-09-19 09:44:59.447714945 +0000 UTC m=+3.214844761 container cleanup 53d58698adb10f08801b1389abdcd2bfdaf1b8ba10f1ef50cc3f50a4a94c9f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:44:59.676 INFO:journalctl@ceph.mds.k.smithi038.stdout:Sep 19 09:44:59 smithi038 bash[71481]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-k 2024-09-19T09:44:59.875 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T09:44:59.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:44:59.908 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T09:44:59.908 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T09:44:59.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.c 2024-09-19T09:45:00.205 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:44:59 smithi078 systemd[1]: Stopping Ceph mds.c for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:45:00.465 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:45:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: 2024-09-19T09:45:00.203+0000 7f5d21532640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:45:00.465 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:45:00 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c[66365]: 2024-09-19T09:45:00.203+0000 7f5d21532640 -1 mds.c *** got signal Terminated *** 2024-09-19T09:45:03.355 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:45:03 smithi078 podman[72313]: 2024-09-19 09:45:03.091014011 +0000 UTC m=+2.994358791 container died cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:45:03.608 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:45:03 smithi078 podman[72313]: 2024-09-19 09:45:03.452939716 +0000 UTC m=+3.356284497 container cleanup cd29df972961e36b67049d4d3af1e881ef4f257b297dfb7fd79f9cf356555de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T09:45:03.608 INFO:journalctl@ceph.mds.c.smithi078.stdout:Sep 19 09:45:03 smithi078 bash[72313]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-c 2024-09-19T09:45:03.899 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T09:45:03.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:45:03.936 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T09:45:03.936 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T09:45:03.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.f 2024-09-19T09:45:04.258 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:04 smithi078 systemd[1]: Stopping Ceph mds.f for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:45:04.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:45:04.256+0000 7f6e8947a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:45:04.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:04 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f[66063]: 2024-09-19T09:45:04.256+0000 7f6e8947a640 -1 mds.f *** got signal Terminated *** 2024-09-19T09:45:07.350 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:07 smithi078 podman[72447]: 2024-09-19 09:45:07.096054697 +0000 UTC m=+2.938630935 container died 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T09:45:07.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:07 smithi078 podman[72447]: 2024-09-19 09:45:07.506789128 +0000 UTC m=+3.349365409 container cleanup 29fae996f6b68a8102e5a097488f4ca98e16e91b5e9e6213a3ef61f834b4c1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T09:45:07.607 INFO:journalctl@ceph.mds.f.smithi078.stdout:Sep 19 09:45:07 smithi078 bash[72447]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-f 2024-09-19T09:45:07.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T09:45:07.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:45:07.892 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T09:45:07.892 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T09:45:07.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.i 2024-09-19T09:45:08.107 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:07 smithi078 systemd[1]: Stopping Ceph mds.i for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:45:08.607 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: 2024-09-19T09:45:08.202+0000 7f25253e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:45:08.607 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:08 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i[66663]: 2024-09-19T09:45:08.202+0000 7f25253e7640 -1 mds.i *** got signal Terminated *** 2024-09-19T09:45:11.312 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:11 smithi078 podman[72566]: 2024-09-19 09:45:11.092133869 +0000 UTC m=+2.983142389 container died f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:45:11.587 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:11 smithi078 podman[72566]: 2024-09-19 09:45:11.401848914 +0000 UTC m=+3.292857433 container cleanup f2f5fc700b906935dab99ba87002846607b66a0c24de58e3e74509001461d165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T09:45:11.588 INFO:journalctl@ceph.mds.i.smithi078.stdout:Sep 19 09:45:11 smithi078 bash[72566]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-i 2024-09-19T09:45:11.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T09:45:11.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:45:11.766 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T09:45:11.766 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T09:45:11.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l 2024-09-19T09:45:12.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:11 smithi078 systemd[1]: Stopping Ceph mds.l for 9a2e9aa4-7668-11ef-bceb-c7b262605968... 2024-09-19T09:45:12.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: 2024-09-19T09:45:12.055+0000 7f3ddcaad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:45:12.107 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:12 smithi078 ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l[70018]: 2024-09-19T09:45:12.055+0000 7f3ddcaad640 -1 mds.l *** got signal Terminated *** 2024-09-19T09:45:15.358 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:15 smithi078 podman[72688]: 2024-09-19 09:45:15.097708343 +0000 UTC m=+3.121318855 container died 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T09:45:15.616 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:15 smithi078 podman[72688]: 2024-09-19 09:45:15.439040489 +0000 UTC m=+3.462650931 container cleanup 199e471199f91d6808ef71e79f02427658045b7cbc2c6d0011ee6d65ce783b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T09:45:15.616 INFO:journalctl@ceph.mds.l.smithi078.stdout:Sep 19 09:45:15 smithi078 bash[72688]: ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968-mds-l 2024-09-19T09:45:15.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a2e9aa4-7668-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T09:45:15.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:45:15.855 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T09:45:15.856 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:45:16.069 INFO:teuthology.orchestra.run.smithi028.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:45:18.702 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:45:18.909 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:45:21.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:45:21.561 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:45:24.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:45:24.053 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:45:24.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:45:24.116 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T09:45:24.869 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi028/crash 2024-09-19T09:45:24.987 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:45:25.018 INFO:teuthology.orchestra.run.smithi028.stderr:tar: /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:45:25.018 INFO:teuthology.orchestra.run.smithi028.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:45:25.067 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi038/crash 2024-09-19T09:45:25.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:45:25.116 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:45:25.116 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:45:25.117 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi078/crash 2024-09-19T09:45:25.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:45:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:45:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:45:25.166 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T09:45:25.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:45:25.219 INFO:teuthology.orchestra.run.smithi028.stdout:2024-09-19T09:35:10.072797+0000 mds.f (mds.3) 2 : cluster [WRN] Scrub error on inode 0x10000001e15 (/volumes/qa/sv_1/ed69f18f-117b-456d-add4-9360702cc391/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/article-18.xml) see mds.f log and `damage ls` output for details 2024-09-19T09:45:25.220 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-19T09:45:25.220 INFO:tasks.cephadm:Compressing logs... 2024-09-19T09:45:25.220 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:45:25.222 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:45:25.225 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:45:25.247 INFO:teuthology.orchestra.run.smithi028.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T09:45:25.247 INFO:teuthology.orchestra.run.smithi028.stderr:: No such file or directory 2024-09-19T09:45:25.247 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T09:45:25.248 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:45:25.249 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:45:25.249 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T09:45:25.249 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:45:25.249 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose --/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.a.log: /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T09:45:25.250 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T09:45:25.250 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T09:45:25.250 INFO:teuthology.orchestra.run.smithi028.stderr: 82.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:45:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T09:45:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T09:45:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T09:45:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T09:45:25.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:45:25.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:45:25.255 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T09:45:25.256 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log: 87.5% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:45:25.257 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T09:45:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T09:45:25.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:45:25.259 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T09:45:25.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:45:25.262 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:45:25.264 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.5.log: /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T09:45:25.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:45:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:45:25.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T09:45:25.270 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:45:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T09:45:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T09:45:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T09:45:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.h.log: 87.5% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:45:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T09:45:25.283 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T09:45:25.283 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:45:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T09:45:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:45:25.313 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:45:25.320 INFO:teuthology.orchestra.run.smithi038.stderr: 94.2% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T09:45:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:45:25.384 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T09:45:25.516 INFO:teuthology.orchestra.run.smithi028.stderr: 94.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T09:45:25.543 INFO:teuthology.orchestra.run.smithi028.stderr: 90.3% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T09:45:25.716 INFO:teuthology.orchestra.run.smithi038.stderr: 91.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T09:45:25.716 INFO:teuthology.orchestra.run.smithi078.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:45:25.801 INFO:teuthology.orchestra.run.smithi078.stderr: 82.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:45:25.804 INFO:teuthology.orchestra.run.smithi078.stderr: 90.5% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:45:25.804 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:45:25.967 INFO:teuthology.orchestra.run.smithi078.stderr: 91.0% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:45:25.974 INFO:teuthology.orchestra.run.smithi038.stderr: 95.3% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T09:45:26.010 INFO:teuthology.orchestra.run.smithi028.stderr: 95.2% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T09:45:26.422 INFO:teuthology.orchestra.run.smithi028.stderr: 90.3% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T09:45:27.485 INFO:teuthology.orchestra.run.smithi078.stderr: 91.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T09:45:30.607 INFO:teuthology.orchestra.run.smithi028.stderr: 94.3% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T09:45:32.891 INFO:teuthology.orchestra.run.smithi028.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T09:45:32.984 INFO:teuthology.orchestra.run.smithi038.stderr: 93.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T09:45:33.494 INFO:teuthology.orchestra.run.smithi028.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T09:45:33.908 INFO:teuthology.orchestra.run.smithi038.stderr: 94.0% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T09:45:34.138 INFO:teuthology.orchestra.run.smithi038.stderr: 94.0% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T09:45:34.139 INFO:teuthology.orchestra.run.smithi028.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T09:45:34.419 INFO:teuthology.orchestra.run.smithi028.stderr: 95.0% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T09:45:34.420 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-19T09:45:34.420 INFO:teuthology.orchestra.run.smithi028.stderr:real 0m9.184s 2024-09-19T09:45:34.421 INFO:teuthology.orchestra.run.smithi028.stderr:user 0m40.132s 2024-09-19T09:45:34.421 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m1.325s 2024-09-19T09:45:34.819 INFO:teuthology.orchestra.run.smithi038.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T09:45:36.453 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T09:45:42.534 INFO:teuthology.orchestra.run.smithi038.stderr: 94.9% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T09:45:42.536 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-19T09:45:42.536 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m17.296s 2024-09-19T09:45:42.536 INFO:teuthology.orchestra.run.smithi038.stderr:user 1m2.528s 2024-09-19T09:45:42.536 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m1.858s 2024-09-19T09:45:50.502 INFO:teuthology.orchestra.run.smithi078.stderr: 94.5% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T09:46:06.065 INFO:teuthology.orchestra.run.smithi078.stderr: 93.9% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T09:46:10.856 INFO:teuthology.orchestra.run.smithi078.stderr: 93.9% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T09:46:11.040 INFO:teuthology.orchestra.run.smithi078.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T09:46:13.394 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T09:46:26.706 INFO:teuthology.orchestra.run.smithi078.stderr: 93.9% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T09:47:00.802 INFO:teuthology.orchestra.run.smithi078.stderr: 94.6% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T09:51:41.704 INFO:teuthology.orchestra.run.smithi078.stderr: 93.7% -- replaced with /var/log/ceph/9a2e9aa4-7668-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T09:51:41.706 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-19T09:51:41.706 INFO:teuthology.orchestra.run.smithi078.stderr:real 6m16.463s 2024-09-19T09:51:41.706 INFO:teuthology.orchestra.run.smithi078.stderr:user 5m10.542s 2024-09-19T09:51:41.706 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m11.499s 2024-09-19T09:51:41.706 INFO:tasks.cephadm:Archiving logs... 2024-09-19T09:51:41.707 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi028/log 2024-09-19T09:51:41.708 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:51:46.614 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi038/log 2024-09-19T09:51:46.615 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:51:55.042 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi078/log 2024-09-19T09:51:55.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:52:44.358 INFO:tasks.cephadm:Removing cluster... 2024-09-19T09:52:44.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force 2024-09-19T09:52:44.572 INFO:teuthology.orchestra.run.smithi028.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:52:44.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force 2024-09-19T09:52:45.086 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:52:45.400 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a2e9aa4-7668-11ef-bceb-c7b262605968 --force 2024-09-19T09:52:46.139 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 9a2e9aa4-7668-11ef-bceb-c7b262605968 2024-09-19T09:52:46.675 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T09:52:46.676 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:52:46.697 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:52:46.717 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:52:46.744 INFO:tasks.cephadm:Teardown complete 2024-09-19T09:52:46.745 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T09:52:46.765 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T09:52:46.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:52:46.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:52:46.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:52:46.919 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T09:52:46.932 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T09:52:46.933 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:52:46.936 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:52:46.939 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:52:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:52:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:52:46.953 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +482us[ +502us] +/- 40ms 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1846us[-1846us] +/- 74ms 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +823us[ +823us] +/- 64ms 2024-09-19T09:52:46.957 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +478us[ +493us] +/- 40ms 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -1839us[-1839us] +/- 74ms 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +816us[ +816us] +/- 64ms 2024-09-19T09:52:46.959 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:52:46.970 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:52:46.971 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-19T09:52:46.971 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +456us[ +427us] +/- 40ms 2024-09-19T09:52:46.971 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1851us[-1851us] +/- 74ms 2024-09-19T09:52:46.971 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +810us[ +782us] +/- 64ms 2024-09-19T09:52:46.971 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:52:46.972 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T09:52:46.986 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T09:52:46.986 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T09:52:47.005 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:52:47.091 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:52:47.167 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:52:47.271 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:52:47.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:47.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:52:47.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:47.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:52:47.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:47.364 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T09:52:47.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T09:52:47.395 INFO:teuthology.task.internal:Duration was 2211.415354 seconds 2024-09-19T09:52:47.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T09:52:47.408 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T09:52:47.408 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:52:47.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:52:47.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:52:47.452 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:52:47.455 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:52:47.484 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:52:47.972 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T09:52:47.972 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T09:52:47.973 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:52:47.998 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T09:52:47.998 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:52:48.043 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-09-19T09:52:48.044 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:52:48.084 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T09:52:48.084 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:52:48.087 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:52:48.090 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:52:48.168 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T09:52:48.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:52:48.171 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:52:48.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:52:49.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T09:52:49.085 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T09:52:49.085 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:52:49.111 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:52:49.139 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:52:49.192 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T09:52:49.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.209 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.230 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-09-19T09:52:49.235 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-09-19T09:52:49.245 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-09-19T09:52:49.287 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:49.302 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:49.334 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:52:49.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:52:49.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T09:52:49.363 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T09:52:49.364 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi028 2024-09-19T09:52:49.364 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:52:49.419 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi038 2024-09-19T09:52:49.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:52:49.458 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911911/remote/smithi078 2024-09-19T09:52:49.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:52:49.494 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T09:52:49.495 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:52:49.498 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:52:49.500 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:52:49.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T09:52:49.556 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T09:52:49.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T09:52:49.572 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T09:52:49.573 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:52:49.575 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:52:49.578 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:52:49.588 INFO:teuthology.orchestra.run.smithi028.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:52 /home/ubuntu/cephtest 2024-09-19T09:52:49.591 INFO:teuthology.orchestra.run.smithi038.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:52 /home/ubuntu/cephtest 2024-09-19T09:52:49.598 INFO:teuthology.orchestra.run.smithi078.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:52 /home/ubuntu/cephtest 2024-09-19T09:52:49.599 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T09:52:49.613 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T09:52:49.648 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2211.415354013443 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-19T09:52:49.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:52:49.731 INFO:teuthology.run:FAIL